-- Logs begin at Sun 2022-09-04 15:24:04 UTC, end at Sun 2022-09-04 16:34:51 UTC. -- Sep 04 15:54:29.207578 dv-cinder-852364-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 04 15:54:29.214879 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 04 15:54:29.216868 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 4 15:54:29 2022] *** Sep 04 15:54:29.216868 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 04 15:54:29.216868 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 04 15:54:29.216868 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: nodename: dv-cinder-852364-6-iscsi Sep 04 15:54:29.216868 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: machine: x86_64 Sep 04 15:54:29.216868 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: clock source: unix Sep 04 15:54:29.216868 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: pcre jit disabled Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: detected number of CPU cores: 4 Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: current working directory: / Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: detected binary path: /usr/bin/uwsgi-core Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: your processes number limit is 47680 Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: your memory page size is 4096 bytes Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: detected max file descriptor number: 2048 Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: lock engine: pthread robust mutexes Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: thunder lock: enabled Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 04 15:54:29.217324 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 04 15:54:29.258363 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: Python main interpreter initialized at 0x556687d259c0 Sep 04 15:54:29.258363 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: python threads support enabled Sep 04 15:54:29.258363 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: your server socket listen backlog is limited to 100 connections Sep 04 15:54:29.258363 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: your mercy for graceful operations on workers is 90 seconds Sep 04 15:54:29.258734 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: mapped 403077 bytes (393 KB) for 2 cores Sep 04 15:54:29.258810 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: *** Operational MODE: preforking *** Sep 04 15:54:29.259087 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: *** uWSGI is running in multiple interpreter mode *** Sep 04 15:54:29.259087 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: spawned uWSGI master process (pid: 81608) Sep 04 15:54:29.259286 dv-cinder-852364-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 04 15:54:29.260159 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: spawned uWSGI worker 1 (pid: 81624, cores: 1) Sep 04 15:54:29.260159 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: spawned uWSGI worker 2 (pid: 81625, cores: 1) Sep 04 15:54:29.261387 dv-cinder-852364-6-iscsi devstack@keystone.service[81608]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 04 15:54:33.590272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 15:54:33.590272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: from cryptography.utils import int_from_bytes Sep 04 15:54:33.591029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 15:54:33.591029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: from cryptography.utils import int_from_bytes Sep 04 15:54:33.593486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 15:54:33.593486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Sep 04 15:54:33.598611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 15:54:33.598611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Sep 04 15:54:33.641833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 15:54:33.642010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 15:54:33.642256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] command line args: [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 15:54:33.642610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 15:54:33.642707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ================================================================================ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 15:54:33.642953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] admin_token = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.643207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_dir = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.643401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.643525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 15:54:33.643613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_source = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.643729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 15:54:33.643813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] debug = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.643921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] command line args: [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 15:54:33.644007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.644183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 15:54:33.644319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.644398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ================================================================================ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 15:54:33.644498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.644593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.644755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.644863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_dir = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.644997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_source = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] debug = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_config_append = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_dir = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.645986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.646879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_config_append = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_dir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.647982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.648954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.649055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.649149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.649248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.649348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.649515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.649636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] publish_errors = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.656988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] standard_threads = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] publish_errors = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_journal = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.657888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_json = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_stderr = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_syslog = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] standard_threads = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.658762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.658900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.658973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.659048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.659048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_journal = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.659184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.659253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_json = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.659322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.659390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_stderr = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.659390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_syslog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.659587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.659665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 15:54:33.659735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.659804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.659872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.659946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.660957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.661934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.662945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663013 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.663943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.664946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.665948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666567 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.666987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.667997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.668889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.669891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.670939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.671909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.672979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.673136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.673213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.673375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.673453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.673634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.673727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.673878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.674925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.675952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.676990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.677950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.678933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.external = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679565 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.679978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.password = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.token = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.680933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.external = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.681070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.681176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.681321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.681432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.681592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.682696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.682803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.682921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.password = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.token = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.683965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.684994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.685942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.686934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.687979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.688895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.689901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.690933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.691994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.693947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694630 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.694880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.695907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.696925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.697988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.698104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.698217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.698329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.698441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.699983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.700903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.701970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.702998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.703951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.704968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.705961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.706910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.707940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.708974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.709974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.710917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.711138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.711311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.711429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.711543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.711656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.711778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.711892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.712005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.712119 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.712232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.712345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.712466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.712580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.712706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.713947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.714060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.714190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.714875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.714991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.715892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.716924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.717987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718567 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.718914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.719944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720630 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.720887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.721966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.722975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.723926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.724995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.725097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.725250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.725355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.725498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.725628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.728484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.728599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.728681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.728750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.728823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.728893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.728964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.729975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730881 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.730952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 15:54:33.731636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.731931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.732072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.732325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.732566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.732815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.733049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.733292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.733535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 15:54:33.733732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 15:54:33.760385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.760613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.760836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.768602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.768602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.768602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.769974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.772664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.799986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.800219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.801478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.801736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.870365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.870538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.923047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.923281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.923755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:33.923998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.090687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.090902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.091128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.091350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.091571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.091790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.092203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.092439 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.092665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.092923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.093105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.093321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.115138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.115364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.115452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.115592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.115672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.115817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.115898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.116036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.116116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.116251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.116342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.116465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.116547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.116680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.133814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.134281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.147120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.147356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.147585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.147665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.147794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.147889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.148874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 15:54:34.171360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.171535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.171619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 15:54:34.171694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 15:54:34.171894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.171978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.172094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 15:54:34.172185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 15:54:34.172372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.172452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.172633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.172712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.172893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.172975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.173152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.173231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.182769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.182860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.183725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.184083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.184192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.184395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.184476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.184744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.191423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.191732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.194343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.194631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.198298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.198298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.198497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.198617 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.199958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.200030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.200102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.200181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.200255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.200328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.200328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.220576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.221585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.221903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.222191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.223050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.223338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.226860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.227151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.227438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.228026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.228314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.228597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.233934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.234226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.235553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.235844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.239926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.240228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.240528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.240822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.241107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.241389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.241661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.241758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.241949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.242030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.242241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.242536 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.242821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.243149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.243467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.243741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.258190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.258485 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.260617 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.260901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.263151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.265978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.266403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.266712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.269192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.269494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.270928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.273681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.277665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.277753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.277828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.277828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.278086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.278378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.278672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.278964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.280605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.280917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.281207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.281495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.281810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.282101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.295294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.295587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.295863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.296142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.298938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.299256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.302904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.302904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.303050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.304536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.306091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.306411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.306696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.309361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.309752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.310041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.310328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.310616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.310926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.311007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.311206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.313934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.314220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.314505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.314791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.315188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.322114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.322402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.322684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.322967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.323354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.324258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.324599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.327549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.327841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.328125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.328407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.328787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.329069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.331650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.340711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.343652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.345996 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.346552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.347972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.348949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.349474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.349785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.350193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.351836 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.353366 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.355417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.355721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.356124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.356576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.358063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.360936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.361338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.361656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.361944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.362542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.363980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.366952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.367344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.367634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.367917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.369713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.370008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.370306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.376775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.377120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.377370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.377498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.383253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.383674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.384485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 15:54:34.384733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.385023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.385318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.385630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.385912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.387233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.387516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.387797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.388097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.388387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.388668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.388943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.389272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 15:54:34.416066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556687d259c0 pid: 81624 (default app) Sep 04 15:54:34.419492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad78d9-216a-4dea-8155-4d65ea3e70a1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:34.419812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad78d9-216a-4dea-8155-4d65ea3e70a1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:34.420059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad78d9-216a-4dea-8155-4d65ea3e70a1 None None] PATH_INFO: `/v3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:34.425037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556687d259c0 pid: 81625 (default app) Sep 04 15:54:34.425169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Sun Sep 4 15:54:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:41.972027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81f46c-f6d1-4a69-9d47-9de93148b6ea None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:41.972638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81f46c-f6d1-4a69-9d47-9de93148b6ea None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:41.972638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81f46c-f6d1-4a69-9d47-9de93148b6ea None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:41.973898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 2/2] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:41.982831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:41.983263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:41.983391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:42.055208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: INFO dbcounter [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] Registered counter for database keystone Sep 04 15:54:42.075016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG oslo_db.sqlalchemy.engines [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 15:54:42.081166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writer thread running {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 15:54:42.261335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81625) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 04 15:54:42.261679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81625) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 04 15:54:42.319687 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:42.321300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:42.448787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1/3] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 04 15:54:42.923143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81f46c-f6d1-4a69-9d47-9de93148b6ea None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:42.993625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: INFO dbcounter [None req-ba81f46c-f6d1-4a69-9d47-9de93148b6ea None None] Registered counter for database keystone Sep 04 15:54:43.016655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG oslo_db.sqlalchemy.engines [None req-ba81f46c-f6d1-4a69-9d47-9de93148b6ea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 15:54:43.023029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writer thread running {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 15:54:43.144917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81f46c-f6d1-4a69-9d47-9de93148b6ea None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:43.149295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f285d5-0817-4d53-aa54-c1cce10ed2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:43.149961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f285d5-0817-4d53-aa54-c1cce10ed2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:43.150248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f285d5-0817-4d53-aa54-c1cce10ed2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:43.150524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f285d5-0817-4d53-aa54-c1cce10ed2ea admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:43.159470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f285d5-0817-4d53-aa54-c1cce10ed2ea admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:43.219380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING keystone.common.rbac_enforcer.enforcer [None req-87f285d5-0817-4d53-aa54-c1cce10ed2ea admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 04 15:54:43.219652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f285d5-0817-4d53-aa54-c1cce10ed2ea admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:43.230358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 3/4] 10.222.0.39 () {58 vars in 1324 bytes} [Sun Sep 4 15:54:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 15:54:43.236358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:43.251133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62ea8e-8238-4dfc-ac35-87b80cbd0160 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:43.255771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c390bc-8cc5-4e57-83dc-2ec6c94c0894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:43.256884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c390bc-8cc5-4e57-83dc-2ec6c94c0894 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:43.257169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c390bc-8cc5-4e57-83dc-2ec6c94c0894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:43.257442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c390bc-8cc5-4e57-83dc-2ec6c94c0894 admin admin] PATH_INFO: `/v3/projects/677669b803a2416fad6430932644a341` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:43.258915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c390bc-8cc5-4e57-83dc-2ec6c94c0894 admin admin] RBAC: Authorizing `identity:get_project(project_id=677669b803a2416fad6430932644a341)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:43.313350 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING keystone.common.rbac_enforcer.enforcer [None req-61c390bc-8cc5-4e57-83dc-2ec6c94c0894 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 04 15:54:43.313720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c390bc-8cc5-4e57-83dc-2ec6c94c0894 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:43.318624 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 2/5] 10.222.0.39 () {56 vars in 1207 bytes} [Sun Sep 4 15:54:43 2022] GET /identity/v3/projects/677669b803a2416fad6430932644a341 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:45.270013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2752a7-2011-4bbc-94db-a4a73bfd1df2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:45.270521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2752a7-2011-4bbc-94db-a4a73bfd1df2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:45.270602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2752a7-2011-4bbc-94db-a4a73bfd1df2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:45.274666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 4/6] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:54:45.280750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d7fef-1ef3-4c31-92b3-2275c26f246e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:45.281148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d7fef-1ef3-4c31-92b3-2275c26f246e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:45.281295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d7fef-1ef3-4c31-92b3-2275c26f246e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:45.301871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d64d7fef-1ef3-4c31-92b3-2275c26f246e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:45.303394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d64d7fef-1ef3-4c31-92b3-2275c26f246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:45.311451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 3/7] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 04 15:54:45.584944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2752a7-2011-4bbc-94db-a4a73bfd1df2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:45.596903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2752a7-2011-4bbc-94db-a4a73bfd1df2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:45.601025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ebede-aeec-4925-b2c9-0e63daaceb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:45.601644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ebede-aeec-4925-b2c9-0e63daaceb8b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:45.601926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ebede-aeec-4925-b2c9-0e63daaceb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:45.602196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ebede-aeec-4925-b2c9-0e63daaceb8b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:45.610312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ebede-aeec-4925-b2c9-0e63daaceb8b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:45.612054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ebede-aeec-4925-b2c9-0e63daaceb8b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:45.621374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 5/8] 10.222.0.39 () {58 vars in 1324 bytes} [Sun Sep 4 15:54:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 15:54:45.626923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d7fef-1ef3-4c31-92b3-2275c26f246e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:45.636328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d7fef-1ef3-4c31-92b3-2275c26f246e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:45.640737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a09b5f-00f7-428c-9f6b-0d05d79b5eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:45.641366 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a09b5f-00f7-428c-9f6b-0d05d79b5eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:45.641684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a09b5f-00f7-428c-9f6b-0d05d79b5eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:45.641960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a09b5f-00f7-428c-9f6b-0d05d79b5eb7 admin admin] PATH_INFO: `/v3/users/f100ee5a590a4764a1d39fd503354ae4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:45.643280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a09b5f-00f7-428c-9f6b-0d05d79b5eb7 admin admin] RBAC: Authorizing `identity:get_user(user_id=f100ee5a590a4764a1d39fd503354ae4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:45.645107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a09b5f-00f7-428c-9f6b-0d05d79b5eb7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:45.646699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 4/9] 10.222.0.39 () {56 vars in 1198 bytes} [Sun Sep 4 15:54:45 2022] GET /identity/v3/users/f100ee5a590a4764a1d39fd503354ae4 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:47.601362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff153-98d0-47b5-92c4-2de5f0aed81d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:47.601886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff153-98d0-47b5-92c4-2de5f0aed81d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:47.601886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff153-98d0-47b5-92c4-2de5f0aed81d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:47.603343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 6/10] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:47.608215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf6a2-80b2-4842-b940-017ff59b63f6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:47.608607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf6a2-80b2-4842-b940-017ff59b63f6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:47.608893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf6a2-80b2-4842-b940-017ff59b63f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:47.627643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7e2bf6a2-80b2-4842-b940-017ff59b63f6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:47.628831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7e2bf6a2-80b2-4842-b940-017ff59b63f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:47.707942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 5/11] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:47.717677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fb5a1-5da8-4072-a360-91f42c76352f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:47.718034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fb5a1-5da8-4072-a360-91f42c76352f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:47.718380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fb5a1-5da8-4072-a360-91f42c76352f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:47.719754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 7/12] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:47.724737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6faf3-e132-4ad5-a886-fc3c5eaa1716 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:47.725038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6faf3-e132-4ad5-a886-fc3c5eaa1716 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:47.725324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6faf3-e132-4ad5-a886-fc3c5eaa1716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:47.744627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c5c6faf3-e132-4ad5-a886-fc3c5eaa1716 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:47.745896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c5c6faf3-e132-4ad5-a886-fc3c5eaa1716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:47.762069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:47.813508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 6/13] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:47.971996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fb5a1-5da8-4072-a360-91f42c76352f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.083725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6faf3-e132-4ad5-a886-fc3c5eaa1716 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.114689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fb5a1-5da8-4072-a360-91f42c76352f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.149895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.150745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.151560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.151902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.162668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:54:48.172295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:54:48.173394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:54:48.174388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 8/14] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:54:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:54:48.179154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.192018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6faf3-e132-4ad5-a886-fc3c5eaa1716 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.215481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:48.226263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.226825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.227110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.227541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.248915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:48.286248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8c203-3ed2-4647-996a-5a1a19eb26cc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.295605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:48.297088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:48.319876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.320496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.320778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.321054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.321759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:54:48.322064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:48.324662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:48.336752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 9/15] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:54:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:48.342564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.396748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 7/16] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:54:48.403032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.436599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:48.442408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f524b-ae66-4591-adee-3166dd156867 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.453921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:48.477431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.478040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.478319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.478592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] PATH_INFO: `/v3/users/f100ee5a590a4764a1d39fd503354ae4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.480851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] RBAC: Authorizing `identity:get_user(user_id=f100ee5a590a4764a1d39fd503354ae4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:48.482372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:48.483948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 10/17] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:54:48 2022] GET /identity/v3/users/f100ee5a590a4764a1d39fd503354ae4 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:48.489584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.504143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9497-9a4d-4951-8552-61803b226192 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.538084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.538716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.538997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.539271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.550749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:48.552263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:54:48.570160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:54:48.571085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:54:48.571880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:54:48.571880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:54:48.571880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:54:48.571880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 8/18] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:54:48.577752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.587879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e700b9-2d0a-4f57-a87e-8424207fadda None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.622183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.622815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.623096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.623368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.624564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:48.626042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:48.627830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 11/19] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:48.633507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:48.664973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:48.679476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15889e57-11d5-4e5e-a41c-9857e8775a8d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.683164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:48.713649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7cd4e-2612-4cc7-85d0-b6cc2d2b5a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.714344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7cd4e-2612-4cc7-85d0-b6cc2d2b5a88 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.714629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7cd4e-2612-4cc7-85d0-b6cc2d2b5a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.714899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7cd4e-2612-4cc7-85d0-b6cc2d2b5a88 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.715545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7cd4e-2612-4cc7-85d0-b6cc2d2b5a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:54:48.715847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7cd4e-2612-4cc7-85d0-b6cc2d2b5a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:48.717229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7cd4e-2612-4cc7-85d0-b6cc2d2b5a88 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:48.734266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e3185-26d5-42aa-a1d4-f24a963b981e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:48.735141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 9/20] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:48.770554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb54ff-296a-4910-9cd1-71ee6b0179f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:48.771175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb54ff-296a-4910-9cd1-71ee6b0179f8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:48.771450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb54ff-296a-4910-9cd1-71ee6b0179f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:48.771719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb54ff-296a-4910-9cd1-71ee6b0179f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:48.772488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb54ff-296a-4910-9cd1-71ee6b0179f8 None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.domain.id=default, user.id=f100ee5a590a4764a1d39fd503354ae4) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:54:48.772791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb54ff-296a-4910-9cd1-71ee6b0179f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:48.774284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb54ff-296a-4910-9cd1-71ee6b0179f8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:48.783532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 12/21] 10.222.0.39 () {56 vars in 1322 bytes} [Sun Sep 4 15:54:48 2022] GET /identity/v3/role_assignments?user.id=f100ee5a590a4764a1d39fd503354ae4&scope.domain.id=default&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:50.816285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1fb5d-6c60-495c-a734-b0a9a132a499 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:50.816765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1fb5d-6c60-495c-a734-b0a9a132a499 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:50.816765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1fb5d-6c60-495c-a734-b0a9a132a499 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:50.818345 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 10/22] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:50.834939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702bab-141a-4880-b0ce-8b314de56f24 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:50.835388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702bab-141a-4880-b0ce-8b314de56f24 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:50.835477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702bab-141a-4880-b0ce-8b314de56f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:50.864312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG passlib.handlers.bcrypt [None req-85702bab-141a-4880-b0ce-8b314de56f24 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81624) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 04 15:54:50.864693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG passlib.handlers.bcrypt [None req-85702bab-141a-4880-b0ce-8b314de56f24 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81624) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 04 15:54:50.918645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-85702bab-141a-4880-b0ce-8b314de56f24 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:50.920255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-85702bab-141a-4880-b0ce-8b314de56f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:51.031008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 13/23] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:51.176054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f401060-7afa-41f7-bd52-032316eb0ff8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:51.176714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f401060-7afa-41f7-bd52-032316eb0ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:51.177201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f401060-7afa-41f7-bd52-032316eb0ff8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:51.178790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 11/24] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:51.191878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e9588-b595-4555-b559-c5e5275c40a2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:51.192164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e9588-b595-4555-b559-c5e5275c40a2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:51.192439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e9588-b595-4555-b559-c5e5275c40a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:51.219187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1b4e9588-b595-4555-b559-c5e5275c40a2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:51.220593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1b4e9588-b595-4555-b559-c5e5275c40a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:51.248320 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:51.310872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 14/25] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:51.477926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f401060-7afa-41f7-bd52-032316eb0ff8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:51.540144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271d7b-ebb5-4aa4-a4aa-729b42981c62 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:51.540424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271d7b-ebb5-4aa4-a4aa-729b42981c62 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:51.540885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271d7b-ebb5-4aa4-a4aa-729b42981c62 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:51.544172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 15/26] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:51.553398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5ddba-6575-4d98-8816-2d86064cae2a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:51.553924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5ddba-6575-4d98-8816-2d86064cae2a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:51.554365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5ddba-6575-4d98-8816-2d86064cae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:51.588765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-60b5ddba-6575-4d98-8816-2d86064cae2a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:51.590238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-60b5ddba-6575-4d98-8816-2d86064cae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:51.640487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f401060-7afa-41f7-bd52-032316eb0ff8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:51.677923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:51.679118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:51.679628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:51.680118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:51.693840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:54:51.701356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:54:51.702267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:54:51.703014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:54:51.705490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 12/27] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:54:51.711548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:51.768653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:51.779259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 16/28] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:51.808359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5ddba-6575-4d98-8816-2d86064cae2a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:51.883982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f950a-b40c-4ab0-aff2-237c0123dd9a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:51.963132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0d2a0-b0e2-4170-be7f-95856e0c8502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:51.965725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0d2a0-b0e2-4170-be7f-95856e0c8502 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:51.967111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0d2a0-b0e2-4170-be7f-95856e0c8502 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:51.967661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0d2a0-b0e2-4170-be7f-95856e0c8502 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:51.969739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0d2a0-b0e2-4170-be7f-95856e0c8502 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:54:51.972658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0d2a0-b0e2-4170-be7f-95856e0c8502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:51.974305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0d2a0-b0e2-4170-be7f-95856e0c8502 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:52.005017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 13/29] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:52.007596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ac48-d4f8-4add-b0e3-2bbb06aa1517 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.008664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ac48-d4f8-4add-b0e3-2bbb06aa1517 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.010017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ac48-d4f8-4add-b0e3-2bbb06aa1517 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.012400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 14/30] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:52.016098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29ac48-d4f8-4add-b0e3-2bbb06aa1517 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:52.020117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:52.045801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5ddba-6575-4d98-8816-2d86064cae2a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:52.089424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:52.090467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.090994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.091484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.092364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:52.094037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:52.161595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:52.180917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29ac48-d4f8-4add-b0e3-2bbb06aa1517 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:52.265776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcabc1-a34d-49cc-a63f-227bff93f96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:52.266791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcabc1-a34d-49cc-a63f-227bff93f96c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.267216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcabc1-a34d-49cc-a63f-227bff93f96c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.267635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcabc1-a34d-49cc-a63f-227bff93f96c None admin] PATH_INFO: `/v3/users/f100ee5a590a4764a1d39fd503354ae4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.271595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcabc1-a34d-49cc-a63f-227bff93f96c None admin] RBAC: Authorizing `identity:get_user(user_id=f100ee5a590a4764a1d39fd503354ae4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:52.275763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcabc1-a34d-49cc-a63f-227bff93f96c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:52.282023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 15/31] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:54:52 2022] GET /identity/v3/users/f100ee5a590a4764a1d39fd503354ae4 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:52.283941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f62-0fc9-43de-9486-a3dadc01ad0b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.284411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f62-0fc9-43de-9486-a3dadc01ad0b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.284861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f62-0fc9-43de-9486-a3dadc01ad0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.340849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a17b5f62-0fc9-43de-9486-a3dadc01ad0b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:52.342168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a17b5f62-0fc9-43de-9486-a3dadc01ad0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:52.477575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 17/32] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Sep 4 15:54:51 2022] POST /identity/v3/roles => generated 219 bytes in 670 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:54:52.487343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:52.522791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 16/33] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:52.524005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5f62-0fc9-43de-9486-a3dadc01ad0b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:52.732747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:52.749407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30f190-fa4d-4736-983e-87f03a369804 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:52.766200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5f62-0fc9-43de-9486-a3dadc01ad0b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:52.808050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea33c70-9213-449b-a297-8b8e90b4f373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:52.808631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea33c70-9213-449b-a297-8b8e90b4f373 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.808914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea33c70-9213-449b-a297-8b8e90b4f373 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.809189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea33c70-9213-449b-a297-8b8e90b4f373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.826281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92bd9d0-6763-4d09-8f33-007ee40d1125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:52.828373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bd9d0-6763-4d09-8f33-007ee40d1125 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.828745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bd9d0-6763-4d09-8f33-007ee40d1125 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.828999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bd9d0-6763-4d09-8f33-007ee40d1125 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.829820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea33c70-9213-449b-a297-8b8e90b4f373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:52.830135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bd9d0-6763-4d09-8f33-007ee40d1125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:52.831429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea33c70-9213-449b-a297-8b8e90b4f373 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:52.831664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bd9d0-6763-4d09-8f33-007ee40d1125 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:52.833283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 17/34] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:52.839148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78987-62a1-42fa-bda8-4098228d3db3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.839148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78987-62a1-42fa-bda8-4098228d3db3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.839148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78987-62a1-42fa-bda8-4098228d3db3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.839148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 18/35] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:52.839148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd248-e453-4739-a1f2-6ec8775b0c1a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.839148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd248-e453-4739-a1f2-6ec8775b0c1a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.839538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd248-e453-4739-a1f2-6ec8775b0c1a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.841333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 19/36] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:52.846760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dd248-e453-4739-a1f2-6ec8775b0c1a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:52.846902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-eea33c70-9213-449b-a297-8b8e90b4f373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:54:52.859322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:54:52.861457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:54:52.866199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:54:52.866199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:54:52.866199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:54:52.866199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 18/37] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:54:52.866199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e844f25-2a81-4d4c-a70d-3647fe567987 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.866199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e844f25-2a81-4d4c-a70d-3647fe567987 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.866199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e844f25-2a81-4d4c-a70d-3647fe567987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.892190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8e844f25-2a81-4d4c-a70d-3647fe567987 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:52.895399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8e844f25-2a81-4d4c-a70d-3647fe567987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:52.969638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 19/38] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:52.970957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648624a9-d530-4970-8e93-ec5cf3442eed None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:52.971307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648624a9-d530-4970-8e93-ec5cf3442eed None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:52.971759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648624a9-d530-4970-8e93-ec5cf3442eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:52.977968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dd248-e453-4739-a1f2-6ec8775b0c1a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:52.997288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-648624a9-d530-4970-8e93-ec5cf3442eed None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:52.998531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-648624a9-d530-4970-8e93-ec5cf3442eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:53.020310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:53.021230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:53.021519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:53.021829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] PATH_INFO: `/v3/domains/default/users/f100ee5a590a4764a1d39fd503354ae4/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:53.023819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f100ee5a590a4764a1d39fd503354ae4, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:53.025292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:53.087356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 20/39] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:53.090797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648624a9-d530-4970-8e93-ec5cf3442eed None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:53.172229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:53.228740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648624a9-d530-4970-8e93-ec5cf3442eed None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:53.238649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 20/40] 10.222.0.39 () {58 vars in 1361 bytes} [Sun Sep 4 15:54:52 2022] PUT /identity/v3/domains/default/users/f100ee5a590a4764a1d39fd503354ae4/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:54:53.245626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:53.282119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:53.311418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:53.312124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:53.312413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:53.312717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:53.313394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:53.314858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:53.393685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 21/41] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Sep 4 15:54:53 2022] POST /identity/v3/roles => generated 225 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:54:53.394918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:53.395497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755929a-1762-4d52-8b03-a7c3ca033cd2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:53.442222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:53.442867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:53.443158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:53.443438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:53.444109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:54:53.444420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:53.446097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:53.464232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 21/42] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:53.464421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:53.539996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7352-c9fa-480d-b02e-78076c981718 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:53.561971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:53.581437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:53.582105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:53.582391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:53.588978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:53.588978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:53.588978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:53.655593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 22/43] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Sep 4 15:54:53 2022] POST /identity/v3/roles => generated 223 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:54:53.656392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:53.668608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:53.710130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb9c75-31cd-4bb7-af73-9f15829031ad None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:53.754495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:53.755258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:53.755547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:53.755925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:53.757234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:53.758875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:53.760934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 22/44] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:53.766652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:53.819309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b06fb-f6e0-4e62-99c1-09f0f6cbe089 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:53.868246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8669837-0da1-4bd4-8bb9-3d46c4f07fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:53.869614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8669837-0da1-4bd4-8bb9-3d46c4f07fd4 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:53.870753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8669837-0da1-4bd4-8bb9-3d46c4f07fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:53.871525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8669837-0da1-4bd4-8bb9-3d46c4f07fd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:53.872748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8669837-0da1-4bd4-8bb9-3d46c4f07fd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:53.876646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8669837-0da1-4bd4-8bb9-3d46c4f07fd4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:53.886705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d8f8-e88b-404e-8765-baa1abc8e74a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:53.927322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23d905-a655-4475-9323-32037eddb53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:53.927889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23d905-a655-4475-9323-32037eddb53d None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:53.928176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23d905-a655-4475-9323-32037eddb53d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:53.928458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23d905-a655-4475-9323-32037eddb53d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:53.929140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23d905-a655-4475-9323-32037eddb53d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:53.930708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23d905-a655-4475-9323-32037eddb53d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:53.945473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:54.029596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 23/45] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:54:53 2022] POST /identity/v3/projects => generated 312 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:54:54.086089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 23/46] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:54:53 2022] POST /identity/v3/projects => generated 301 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:54:55.261517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2116df8-d2e1-4f41-baa4-520c1608ed61 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:55.262051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2116df8-d2e1-4f41-baa4-520c1608ed61 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:55.262135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2116df8-d2e1-4f41-baa4-520c1608ed61 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:55.263482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 24/47] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:55.268829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4eb3a-3d32-4826-8068-96209bd56da5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:55.269257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4eb3a-3d32-4826-8068-96209bd56da5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:55.269390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4eb3a-3d32-4826-8068-96209bd56da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:55.292457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-86a4eb3a-3d32-4826-8068-96209bd56da5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:55.293783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-86a4eb3a-3d32-4826-8068-96209bd56da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:55.316800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:55.361778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 24/48] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:55.632815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2116df8-d2e1-4f41-baa4-520c1608ed61 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:55.733410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2116df8-d2e1-4f41-baa4-520c1608ed61 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:55.767592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:55.768224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:55.768513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:55.768791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:55.778843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:54:55.783794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:54:55.784620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:54:55.785406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 25/49] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:54:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:54:55.790684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4eb3a-3d32-4826-8068-96209bd56da5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:55.890038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4eb3a-3d32-4826-8068-96209bd56da5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:55.926993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a2a9-81a6-4444-be15-910bcd8a6b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:55.927622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a2a9-81a6-4444-be15-910bcd8a6b7f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:55.927906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a2a9-81a6-4444-be15-910bcd8a6b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:55.928184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a2a9-81a6-4444-be15-910bcd8a6b7f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:55.928860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a2a9-81a6-4444-be15-910bcd8a6b7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:54:55.929167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a2a9-81a6-4444-be15-910bcd8a6b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:55.930587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a2a9-81a6-4444-be15-910bcd8a6b7f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:55.939575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 25/50] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:54:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:55.945717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:55.960517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5d38a-87b4-4498-8181-da2f7c167167 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:55.960810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5d38a-87b4-4498-8181-da2f7c167167 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:55.961089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5d38a-87b4-4498-8181-da2f7c167167 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:55.962434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 26/51] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:55.967319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43523d-c852-4d18-a6e9-6305ce08fb41 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:55.967605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43523d-c852-4d18-a6e9-6305ce08fb41 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:55.967884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43523d-c852-4d18-a6e9-6305ce08fb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:55.987139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9e43523d-c852-4d18-a6e9-6305ce08fb41 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:55.988328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9e43523d-c852-4d18-a6e9-6305ce08fb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:56.001702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:56.005225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:56.048598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e23b5-9d90-4966-ba47-77e68f4c329d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:56.056125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 27/52] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:54:56.082951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:56.083591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:56.083876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:56.084156 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] PATH_INFO: `/v3/users/f100ee5a590a4764a1d39fd503354ae4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:56.085289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] RBAC: Authorizing `identity:get_user(user_id=f100ee5a590a4764a1d39fd503354ae4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:56.086800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:56.089838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 26/53] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:54:55 2022] GET /identity/v3/users/f100ee5a590a4764a1d39fd503354ae4 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:56.094128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43523d-c852-4d18-a6e9-6305ce08fb41 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:56.190089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43523d-c852-4d18-a6e9-6305ce08fb41 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:56.223814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:56.224446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:56.224728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:56.225001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:56.226135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:56.227558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:56.229838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 28/54] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:56.235437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:56.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3636870-e971-493c-b34c-b6c3f5880622 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:56.336899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:56.356251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:56.359556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:56.370299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:56.370948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:56.371228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:56.371505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:56.372242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.domain.id=default, user.id=f100ee5a590a4764a1d39fd503354ae4) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:54:56.372549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:56.374046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:56.381636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 27/55] 10.222.0.39 () {56 vars in 1322 bytes} [Sun Sep 4 15:54:56 2022] GET /identity/v3/role_assignments?user.id=f100ee5a590a4764a1d39fd503354ae4&scope.domain.id=default&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:56.442867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596669-4b1e-4177-b046-7ac24c20939c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:56.480359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193b744-94a6-42bc-a459-e394d5fc734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:56.481025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193b744-94a6-42bc-a459-e394d5fc734b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:56.481303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193b744-94a6-42bc-a459-e394d5fc734b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:56.481595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193b744-94a6-42bc-a459-e394d5fc734b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:56.482747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193b744-94a6-42bc-a459-e394d5fc734b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:56.484205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193b744-94a6-42bc-a459-e394d5fc734b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:56.485857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 29/56] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:54:56.493215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:56.601736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db39894-53e9-460c-95fd-23406319ea1b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:56.634908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df1523-d173-467b-ae57-a7acbf7e9c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:56.636581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1523-d173-467b-ae57-a7acbf7e9c95 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:56.636870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1523-d173-467b-ae57-a7acbf7e9c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:56.637155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1523-d173-467b-ae57-a7acbf7e9c95 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:56.637847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df1523-d173-467b-ae57-a7acbf7e9c95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:56.639307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df1523-d173-467b-ae57-a7acbf7e9c95 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:56.742033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 28/57] 10.222.0.39 () {60 vars in 1140 bytes} [Sun Sep 4 15:54:56 2022] POST /identity/v3/projects => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:54:58.662564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c45b7f-a32b-47ee-8c29-83d50c0b6e42 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:58.663049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c45b7f-a32b-47ee-8c29-83d50c0b6e42 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:58.663049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c45b7f-a32b-47ee-8c29-83d50c0b6e42 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:58.664388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 30/58] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:54:58.669666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603833-81c9-4e92-9de4-99586bd8f913 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:58.670024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603833-81c9-4e92-9de4-99586bd8f913 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:58.670238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603833-81c9-4e92-9de4-99586bd8f913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:58.689552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f1603833-81c9-4e92-9de4-99586bd8f913 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:54:58.695651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1603833-81c9-4e92-9de4-99586bd8f913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:54:58.719744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:58.756551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 29/59] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:54:59.034307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c45b7f-a32b-47ee-8c29-83d50c0b6e42 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:59.122305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:54:59.135393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c45b7f-a32b-47ee-8c29-83d50c0b6e42 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:59.175488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcde7bb-774e-4aa7-ae7c-78bf938e2e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:59.176128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde7bb-774e-4aa7-ae7c-78bf938e2e0e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:59.176412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde7bb-774e-4aa7-ae7c-78bf938e2e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:59.176689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde7bb-774e-4aa7-ae7c-78bf938e2e0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:59.177831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcde7bb-774e-4aa7-ae7c-78bf938e2e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:59.179310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcde7bb-774e-4aa7-ae7c-78bf938e2e0e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:59.180865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 31/60] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:54:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:54:59.186684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1603833-81c9-4e92-9de4-99586bd8f913 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:54:59.283082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1603833-81c9-4e92-9de4-99586bd8f913 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:54:59.319352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b358c-c6bc-442e-9c32-20ab1edead18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:54:59.320022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b358c-c6bc-442e-9c32-20ab1edead18 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:54:59.320307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b358c-c6bc-442e-9c32-20ab1edead18 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:54:59.320589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b358c-c6bc-442e-9c32-20ab1edead18 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:54:59.321265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b358c-c6bc-442e-9c32-20ab1edead18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:54:59.323270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b358c-c6bc-442e-9c32-20ab1edead18 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:54:59.432297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 30/61] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:54:59 2022] POST /identity/v3/users => generated 275 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:02.418969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228ab26-8513-4c21-bf21-99ef41e8c1e6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:02.419993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228ab26-8513-4c21-bf21-99ef41e8c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:02.420491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228ab26-8513-4c21-bf21-99ef41e8c1e6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:02.422062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 32/62] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:02.427865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841727e-ea1e-4cdd-b5c3-6629ade84db8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:02.428703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841727e-ea1e-4cdd-b5c3-6629ade84db8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:02.429270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841727e-ea1e-4cdd-b5c3-6629ade84db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:02.458581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4841727e-ea1e-4cdd-b5c3-6629ade84db8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:02.460133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4841727e-ea1e-4cdd-b5c3-6629ade84db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:02.485825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:02.565291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 31/63] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:02.695955 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce3547-5216-4f59-8c67-6b156ef32fc2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:02.696562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce3547-5216-4f59-8c67-6b156ef32fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:02.697038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce3547-5216-4f59-8c67-6b156ef32fc2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:02.699340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 33/64] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:02.704883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68854da2-17fe-494e-9c52-a6250790d8a3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:02.705476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68854da2-17fe-494e-9c52-a6250790d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:02.706000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68854da2-17fe-494e-9c52-a6250790d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:02.730173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84648a-6626-4ec1-9017-d08f90d229a4 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:02.730470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84648a-6626-4ec1-9017-d08f90d229a4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:02.730740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84648a-6626-4ec1-9017-d08f90d229a4 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:02.731864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-68854da2-17fe-494e-9c52-a6250790d8a3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:02.732350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 34/65] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:02.733515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68854da2-17fe-494e-9c52-a6250790d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:02.737547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1091c2b-7358-43b2-9ae2-550e4a11f4b5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:02.737868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1091c2b-7358-43b2-9ae2-550e4a11f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:02.738152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1091c2b-7358-43b2-9ae2-550e4a11f4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:02.764355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c1091c2b-7358-43b2-9ae2-550e4a11f4b5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:02.766847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1091c2b-7358-43b2-9ae2-550e4a11f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:02.844841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 35/66] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:02.866534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 32/67] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:02.939457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1091c2b-7358-43b2-9ae2-550e4a11f4b5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.029187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:03.049184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1091c2b-7358-43b2-9ae2-550e4a11f4b5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.094259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281402c-9633-48a8-b317-5a19d8af2672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.094914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281402c-9633-48a8-b317-5a19d8af2672 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.095200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281402c-9633-48a8-b317-5a19d8af2672 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.095476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281402c-9633-48a8-b317-5a19d8af2672 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.108311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281402c-9633-48a8-b317-5a19d8af2672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2281402c-9633-48a8-b317-5a19d8af2672 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:03.113226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:03.114113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:03.114867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 36/68] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:03.121477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68854da2-17fe-494e-9c52-a6250790d8a3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.152757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d18d7-a69f-4309-9742-63f31e61c7f4 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.153086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d18d7-a69f-4309-9742-63f31e61c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.153320 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d18d7-a69f-4309-9742-63f31e61c7f4 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.155278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 37/69] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:03.156039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d18d7-a69f-4309-9742-63f31e61c7f4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.234774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:03.255193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68854da2-17fe-494e-9c52-a6250790d8a3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.269177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d18d7-a69f-4309-9742-63f31e61c7f4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.292089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eef733-2575-41a2-9a50-cc7351111f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.292639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eef733-2575-41a2-9a50-cc7351111f55 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.292936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eef733-2575-41a2-9a50-cc7351111f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.293196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eef733-2575-41a2-9a50-cc7351111f55 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.293908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eef733-2575-41a2-9a50-cc7351111f55 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:03.294221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eef733-2575-41a2-9a50-cc7351111f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:03.295614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eef733-2575-41a2-9a50-cc7351111f55 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:03.304393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 33/70] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:03.305607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd31eca-1bd3-45d7-ad72-8e084949e7cb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.305938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd31eca-1bd3-45d7-ad72-8e084949e7cb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.306219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd31eca-1bd3-45d7-ad72-8e084949e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.310219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.310875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.311162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.311444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.329103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5bd31eca-1bd3-45d7-ad72-8e084949e7cb None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:03.329284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:03.330327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5bd31eca-1bd3-45d7-ad72-8e084949e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:03.334168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:03.335235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 38/71] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:03.336495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.351148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:03.399822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 34/72] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:03.402331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60886f9-4fd7-4868-a9d0-4ee6bc30ab68 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.402693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60886f9-4fd7-4868-a9d0-4ee6bc30ab68 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.402979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60886f9-4fd7-4868-a9d0-4ee6bc30ab68 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.404399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 35/73] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:03.406360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60886f9-4fd7-4868-a9d0-4ee6bc30ab68 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.443412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96fb06-c59a-4df1-abb9-0b9f952eeb7d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.477656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.478294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.479190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.479678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.490825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:03.495769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:03.496644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 39/74] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:03.498159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.499662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:03.529250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60886f9-4fd7-4868-a9d0-4ee6bc30ab68 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.565551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:03.567590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a963b-5669-4ae7-9bb8-299bdf291ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.568235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a963b-5669-4ae7-9bb8-299bdf291ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.568522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a963b-5669-4ae7-9bb8-299bdf291ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.568799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a963b-5669-4ae7-9bb8-299bdf291ca4 None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.583788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a963b-5669-4ae7-9bb8-299bdf291ca4 None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:03.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a963b-5669-4ae7-9bb8-299bdf291ca4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:03.592072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 36/75] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:03.592072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f470e7-f88a-485a-b8c7-c4a556a0ba27 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.592072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f470e7-f88a-485a-b8c7-c4a556a0ba27 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.592072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f470e7-f88a-485a-b8c7-c4a556a0ba27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.606408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8bc24-e60f-4c2d-ae80-ce010d376e5f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.609078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-77f470e7-f88a-485a-b8c7-c4a556a0ba27 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:03.610305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-77f470e7-f88a-485a-b8c7-c4a556a0ba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:03.642820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.643437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.643750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.644005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.645478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:03.645478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:03.646423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:03.658615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 40/76] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:03.659248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.687628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 37/77] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:03.688485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f470e7-f88a-485a-b8c7-c4a556a0ba27 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.757312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:03.777814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99370128-51c3-4a82-98eb-0e1aaa8871de None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.797871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f470e7-f88a-485a-b8c7-c4a556a0ba27 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.805052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:03.813013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.813906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.814200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.814479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.817245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:03.817245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:03.817245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:03.830524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 41/78] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:03.830524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.842197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.842767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.843054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.843329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.922970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:03.924464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:03.940742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2fd81-85b3-485a-a201-f766bd67a6a9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:03.975768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:03.976409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:03.976695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:03.976977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:03.978389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:03.980664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:03.982287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 42/79] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:03.983057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:03.990105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.032194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 38/80] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:04.033041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.048591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.093074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c18eb5-f045-4470-87a6-ab59333628f6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.127304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.127942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.128223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.128497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.137188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e827cbe-9a8c-4aa1-98e0-cc2ce0f4a381 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.138602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:04.143918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:04.144854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:04.145627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 43/81] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:04.146260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.172519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.173261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.173445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.173747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] PATH_INFO: `/v3/users/f100ee5a590a4764a1d39fd503354ae4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.174888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] RBAC: Authorizing `identity:get_user(user_id=f100ee5a590a4764a1d39fd503354ae4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.176338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.177937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 39/82] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/users/f100ee5a590a4764a1d39fd503354ae4 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:04.178773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.209105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.247212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bdf05-f5ad-4ae8-9902-8032f2eefbae None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.263585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.280027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f1b19-d822-4521-aa0b-f7d7bd49ff29 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.280284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.280922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.281205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.281480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.282628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.284071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.285641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 44/83] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:04.287950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.314325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.314943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.315257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.315535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.385207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.386755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.389813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9d770-5f21-4eee-8820-9957e8b3b860 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.414504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.423315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.427780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.427780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.427780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.427780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.427780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.428946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 45/84] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:04.429497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.484550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 40/85] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:04.485679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.488405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.533114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a0202-cc73-46fa-8d84-e6376392a212 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.566366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.566974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.567258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.567531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.568189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:04.568492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.569890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.577886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 46/86] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:04.578650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.590558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1709f80-5673-4ecb-80fe-0bdbde5cbd31 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.619742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.625646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.626535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.626821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.627102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.679037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0deeb-9c69-4b2e-bf64-f71210815028 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.694014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.695288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.695498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.712788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.713999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.714288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.714570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.715223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.716671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.793156 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 41/87] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:04.794024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.804166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 47/88] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:04 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:04.805151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.832417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:04.917839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085eb0-dc9e-4c8b-aab8-6b041454cff3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.920863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389f0d9-d38b-479a-b227-8d82aa1d891c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:04.959080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.959657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.959937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.960211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.960671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:04.961287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:04.961509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.961709 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:04.961993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:04.962728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=300553fe41194f45839b5a1043768774, user.id=dce6be918e8741208b14eb3247cf7e38) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:04.963100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.963539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:04.964738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 48/89] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:04.964982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:04.965503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:04.976651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 42/90] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/role_assignments?user.id=dce6be918e8741208b14eb3247cf7e38&scope.project.id=300553fe41194f45839b5a1043768774&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:04.977506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:05.018208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:05.113546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee61ca-b898-46e9-ba09-72fb04ebde17 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:05.117398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:05.127042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed178d-c9be-4432-94b4-38fb722977fd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:05.169919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:05.170817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:05.171505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:05.171789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:05.173306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:05.174731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:05.176542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 49/91] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:05.176649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:05.177042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:05.177142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:05.177335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:05.178403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:05.179577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:05.181890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:05.183577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 43/92] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:04 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:05.189155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:05.308899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d313c0-fd39-4cb0-aec8-f8606cd4cdc2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:05.315762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73240-5687-4fea-a00a-e855ce6eea94 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:05.328882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:05.352695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:05.353362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:05.354275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:05.354566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:05.363618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:05.364247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:05.364534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:05.364815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:05.365582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] RBAC: Adding query filter params (role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=300553fe41194f45839b5a1043768774, user.id=dce6be918e8741208b14eb3247cf7e38) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:05.365902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:05.367935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:05.377861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 50/93] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:05 2022] GET /identity/v3/role_assignments?user.id=dce6be918e8741208b14eb3247cf7e38&scope.project.id=300553fe41194f45839b5a1043768774&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:05.378709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:05.409826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:05.484342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:05.485925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:05.579709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42516-cb9d-4bd2-a681-419f48d56001 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:05.621730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:05.622657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:05.622949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:05.623227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:05.624234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=300553fe41194f45839b5a1043768774, user.id=f100ee5a590a4764a1d39fd503354ae4) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:05.624551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:05.626052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:05.633273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 51/94] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:05 2022] GET /identity/v3/role_assignments?user.id=f100ee5a590a4764a1d39fd503354ae4&scope.project.id=300553fe41194f45839b5a1043768774&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:05.687860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 44/95] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:05.693477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:05.745237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:05.831281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850b761-206f-4a85-af1d-83a2fb86128c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:05.868813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6558d-92cb-403c-8435-3431f50e1706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:05.869526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6558d-92cb-403c-8435-3431f50e1706 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:05.869849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6558d-92cb-403c-8435-3431f50e1706 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:05.870133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6558d-92cb-403c-8435-3431f50e1706 None admin] PATH_INFO: `/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:05.871293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6558d-92cb-403c-8435-3431f50e1706 None admin] RBAC: Authorizing `identity:get_project(project_id=0e2b8dd5a93f4b479d8bc81084d2a51c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:05.872751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6558d-92cb-403c-8435-3431f50e1706 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:05.874433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 52/96] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:05 2022] GET /identity/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:05.880693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:05.974026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:06.027252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e623fb-cef0-4117-b121-0f4a2563302d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:06.067639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943c18-52f3-4c58-99b7-c5c75aebc688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:06.068266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943c18-52f3-4c58-99b7-c5c75aebc688 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:06.068550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943c18-52f3-4c58-99b7-c5c75aebc688 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:06.068827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943c18-52f3-4c58-99b7-c5c75aebc688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:06.069558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943c18-52f3-4c58-99b7-c5c75aebc688 None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=0e2b8dd5a93f4b479d8bc81084d2a51c, user.id=dce6be918e8741208b14eb3247cf7e38) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:06.069889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943c18-52f3-4c58-99b7-c5c75aebc688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:06.071354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943c18-52f3-4c58-99b7-c5c75aebc688 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:06.082478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 45/97] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:05 2022] GET /identity/v3/role_assignments?user.id=dce6be918e8741208b14eb3247cf7e38&scope.project.id=0e2b8dd5a93f4b479d8bc81084d2a51c&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:07.634098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8261ee2-ee6e-43b2-b550-62b1a1f525a2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:07.635634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8261ee2-ee6e-43b2-b550-62b1a1f525a2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:07.636123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8261ee2-ee6e-43b2-b550-62b1a1f525a2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:07.637714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 53/98] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:07.643177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786f7d-51f0-4fde-a5dc-31f0f9ae3a14 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:07.643624 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786f7d-51f0-4fde-a5dc-31f0f9ae3a14 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:07.643795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786f7d-51f0-4fde-a5dc-31f0f9ae3a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:07.666038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6e786f7d-51f0-4fde-a5dc-31f0f9ae3a14 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:07.667232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e786f7d-51f0-4fde-a5dc-31f0f9ae3a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:07.757005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 46/99] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:08.034576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec696c4-a4cb-4290-a1f4-2568e460d818 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.034942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec696c4-a4cb-4290-a1f4-2568e460d818 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.035042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec696c4-a4cb-4290-a1f4-2568e460d818 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.036563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 54/100] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:08.051549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5f861-e592-4f60-a12b-4eeefd03d130 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.051859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5f861-e592-4f60-a12b-4eeefd03d130 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.052118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5f861-e592-4f60-a12b-4eeefd03d130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.073633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-96a5f861-e592-4f60-a12b-4eeefd03d130 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:08.075693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-96a5f861-e592-4f60-a12b-4eeefd03d130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:08.141924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:08.162847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 47/101] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:08.194937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3a58f-ffbe-438e-8423-23d7ecbc7349 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.195236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3a58f-ffbe-438e-8423-23d7ecbc7349 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.195500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3a58f-ffbe-438e-8423-23d7ecbc7349 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.196880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 55/102] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:08.203917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339095-bca1-4c7b-9398-f39dcd54c1f1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.204208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339095-bca1-4c7b-9398-f39dcd54c1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.204486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339095-bca1-4c7b-9398-f39dcd54c1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.207791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7f758-3b4c-4962-859b-6d399a8b90a8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.208147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7f758-3b4c-4962-859b-6d399a8b90a8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.208433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7f758-3b4c-4962-859b-6d399a8b90a8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.209741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 56/103] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:08.215156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ce0f6-37a6-4158-9e9a-af4da0b476bb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.215448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ce0f6-37a6-4158-9e9a-af4da0b476bb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.215723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ce0f6-37a6-4158-9e9a-af4da0b476bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.244906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-13339095-bca1-4c7b-9398-f39dcd54c1f1 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:08.246114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-13339095-bca1-4c7b-9398-f39dcd54c1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:08.246328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-689ce0f6-37a6-4158-9e9a-af4da0b476bb None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:08.247664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-689ce0f6-37a6-4158-9e9a-af4da0b476bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:08.346122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 57/104] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:08.347589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9802b1-4b83-4612-945c-41866a13b294 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.348087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9802b1-4b83-4612-945c-41866a13b294 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.348597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9802b1-4b83-4612-945c-41866a13b294 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.350111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 58/105] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:08.350960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9802b1-4b83-4612-945c-41866a13b294 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:08.354304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 48/106] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:08.359989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5d83f-103e-49d0-bce7-f1d9a4b5b816 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.359989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5d83f-103e-49d0-bce7-f1d9a4b5b816 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.359989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5d83f-103e-49d0-bce7-f1d9a4b5b816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.376712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b4c5d83f-103e-49d0-bce7-f1d9a4b5b816 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:08.378200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b4c5d83f-103e-49d0-bce7-f1d9a4b5b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:08.428259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:08.491218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 49/107] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:08.492095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5d83f-103e-49d0-bce7-f1d9a4b5b816 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:08.557982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:08.559908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9802b1-4b83-4612-945c-41866a13b294 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:08.636653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5d83f-103e-49d0-bce7-f1d9a4b5b816 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:08.646351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:08.647003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.647286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.647560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.666672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:08.671689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:08.672509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:08.673259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:08.674805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 59/108] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:08.686555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:08.715186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:08.715872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.716125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.716405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.735197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:08.740218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:08.741103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:08.741887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 50/109] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:08.742990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:08.816126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:08.824673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be58c-24da-4bc7-bad1-568e854d96e6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:08.845152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66393a27-0f2e-4af8-a6f1-7a492e88ece0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:08.854588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:08.862912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:08.863533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.863849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.864085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.864769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:08.865047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:08.866457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:08.880403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 60/110] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:08.880403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:08.887227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:08.887939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:08.888221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:08.888497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:08.900685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:08.907970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:08.908816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 51/111] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:08.914578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:08.981059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ddcaf-6b93-41d1-bb4c-3f5a555572e1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.017194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.017861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.018145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.018417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.019556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.021002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.022593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 61/112] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.022593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.030574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415554cc-e363-40a7-9e6a-d1f74ecd1812 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.049021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.066743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.067363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.067657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.067933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.068596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:09.068901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.070310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.081885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 52/113] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.083548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.112027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.129940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10716590-b399-40e5-91ee-c2e35d6e7a94 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.171794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.172359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.172644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.172923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.183725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:09.188596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:09.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:09.190380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 62/114] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:09.191132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.196251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6328c0-1454-42c5-a565-b2d24da07c30 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.232742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.233395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.233721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.234004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.235162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.236616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.238261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 53/115] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.239007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.283868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.299456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d781e7-34b6-4eef-a58c-1b7d4d0a5715 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.336229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.336851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.336963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.337133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.337409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.338686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:09.338994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.340370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.347885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229749-4571-4028-b8dc-35fec35e33a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.348564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 63/116] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.353556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.382823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.383437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.383723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.384001 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.385356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.386926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.449518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64481e-5351-4226-8243-d3133e6367bb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.477088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 54/117] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:55:09 2022] POST /identity/v3/projects => generated 302 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:09.477088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.491494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.492190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.492456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.492735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] PATH_INFO: `/v3/users/263efbc979c844daa8077f6274bc6521` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.514457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] RBAC: Authorizing `identity:get_user(user_id=263efbc979c844daa8077f6274bc6521)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.515893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.522583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 64/118] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/users/263efbc979c844daa8077f6274bc6521 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.522583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.527451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.599955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b93e85-233a-4c86-91d4-689499ea0bf0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.641552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2fc9d-60d9-4ed3-b421-b0b1185315b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.642482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2fc9d-60d9-4ed3-b421-b0b1185315b2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.642768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2fc9d-60d9-4ed3-b421-b0b1185315b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.643046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2fc9d-60d9-4ed3-b421-b0b1185315b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.643732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2fc9d-60d9-4ed3-b421-b0b1185315b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:09.644040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2fc9d-60d9-4ed3-b421-b0b1185315b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.645438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2fc9d-60d9-4ed3-b421-b0b1185315b2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.647946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e71b8-8ee0-4b32-b384-64af89c4100b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.655778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 55/119] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.656314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76afe7-bb26-454f-b9ee-0e686eac56d7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.656679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76afe7-bb26-454f-b9ee-0e686eac56d7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.656963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76afe7-bb26-454f-b9ee-0e686eac56d7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.662979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 56/120] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:09.662979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76afe7-bb26-454f-b9ee-0e686eac56d7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.673671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.690586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.691237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.691528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.691806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.692980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.694456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.696414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 65/121] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.697762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.772261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76afe7-bb26-454f-b9ee-0e686eac56d7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.788505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.808140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.808796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.809081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.809358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] PATH_INFO: `/v3/users/f100ee5a590a4764a1d39fd503354ae4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.810513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] RBAC: Authorizing `identity:get_user(user_id=f100ee5a590a4764a1d39fd503354ae4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.812084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.813679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 57/122] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/users/f100ee5a590a4764a1d39fd503354ae4 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.813679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:09.829546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a10dd1-6322-4dbd-8cdd-8eeea65dd097 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.866500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.867157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.867422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.867674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.901365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:09.923358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614037f-41d9-488c-8700-a0400e251c1e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:09.942534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.944127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.963807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a19b81-0b52-46eb-a6d8-796cd2126d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:09.964509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a19b81-0b52-46eb-a6d8-796cd2126d7e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.964801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a19b81-0b52-46eb-a6d8-796cd2126d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.965085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a19b81-0b52-46eb-a6d8-796cd2126d7e None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.966256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a19b81-0b52-46eb-a6d8-796cd2126d7e None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:09.967710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a19b81-0b52-46eb-a6d8-796cd2126d7e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:09.969606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 58/123] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:09.970619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45b3d-8488-4988-8a8e-adacc7afe766 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:09.970912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45b3d-8488-4988-8a8e-adacc7afe766 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:09.971193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45b3d-8488-4988-8a8e-adacc7afe766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:09.989057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-41d45b3d-8488-4988-8a8e-adacc7afe766 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:09.990311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-41d45b3d-8488-4988-8a8e-adacc7afe766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:10.026077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:10.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 66/124] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:10.057070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.076425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 59/125] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:10.076425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45b3d-8488-4988-8a8e-adacc7afe766 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.166904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:10.194868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45b3d-8488-4988-8a8e-adacc7afe766 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.204450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347c6e4-1b12-4877-bf2c-11739f4bf636 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.239956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.240363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.240628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.240916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.241011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.241195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.241287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.241553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774/users/dce6be918e8741208b14eb3247cf7e38/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.242447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:10.243595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=300553fe41194f45839b5a1043768774, user_id=dce6be918e8741208b14eb3247cf7e38, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:10.244557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:10.245709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:10.246393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 60/126] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:10 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:10.247390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.325911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 67/127] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:10 2022] PUT /identity/v3/projects/300553fe41194f45839b5a1043768774/users/dce6be918e8741208b14eb3247cf7e38/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:10.325911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.366921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:10.380969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a28f1-a421-4a50-aa62-d4df4b61b0ba None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.431256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.432400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.432882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.433350 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.434827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:10.436555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:10.438809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 61/128] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:10 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:10.439890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.460442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:10.483768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b32aa5-c013-4d50-ac2f-5b65e86c72f2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.524124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.524830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.525161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.525589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.526759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:10.528282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:10.530258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 68/129] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:10 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:10.531042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.574425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629ad3d-2bf2-482a-b7c7-e4ac0cfe85e4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.612246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.613315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.613841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.614317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774/users/f100ee5a590a4764a1d39fd503354ae4/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.616637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=300553fe41194f45839b5a1043768774, user_id=f100ee5a590a4764a1d39fd503354ae4, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:10.618538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:10.641172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:10.644050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeda77-0fac-424d-9093-e0262488788a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.678612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.679239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.679625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.679858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.691082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:10.695822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:10.696682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 69/130] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:10.699298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.708754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 62/131] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:10 2022] PUT /identity/v3/projects/300553fe41194f45839b5a1043768774/users/f100ee5a590a4764a1d39fd503354ae4/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:10.710898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:10.794814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:10.840458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367e2f4-5a91-47b1-9fb3-8c2c1bb7a5d3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.850226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf103-6ac6-4579-9077-dfc5ab81061b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:10.915033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.916137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.916422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.916697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.917451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] RBAC: Adding query filter params (role.id=7b93900f6d71493ca4954f9eba5d159b, scope.project.id=300553fe41194f45839b5a1043768774, user.id=263efbc979c844daa8077f6274bc6521) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:10.917785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:10.919258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:10.923451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:10.923451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:10.923770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:10.924013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774/users/dce6be918e8741208b14eb3247cf7e38/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:10.926337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=300553fe41194f45839b5a1043768774, user_id=dce6be918e8741208b14eb3247cf7e38, role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:10.927788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:10.935787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 63/132] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:10 2022] GET /identity/v3/role_assignments?user.id=263efbc979c844daa8077f6274bc6521&scope.project.id=300553fe41194f45839b5a1043768774&role.id=7b93900f6d71493ca4954f9eba5d159b => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:10.936882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:11.003689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 70/133] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:10 2022] PUT /identity/v3/projects/300553fe41194f45839b5a1043768774/users/dce6be918e8741208b14eb3247cf7e38/roles/eb0cba906bfe429883619da6e985a2b4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:11.090425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccb304-3674-4a67-9518-4310e18e46dd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:11.131795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:11.132762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:11.133241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:11.133760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:11.135186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:11.135686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:11.137260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:11.142960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:11.152920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 64/134] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:11.159632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:11.188184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:11.278275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1c48-2dca-43a5-a8c3-26258d8438a1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:11.317987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:11.318978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:11.319482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:11.319957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:11.321286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:11.323714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:11.325513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 71/135] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:11 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:11.336279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:11.445702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1937-d003-40e2-be1b-a639e25c0ef3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:11.486299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ce150-75d7-4032-8e6b-a60c4541599a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:11.487357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce150-75d7-4032-8e6b-a60c4541599a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:11.487866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce150-75d7-4032-8e6b-a60c4541599a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:11.488336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce150-75d7-4032-8e6b-a60c4541599a None admin] PATH_INFO: `/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:11.489933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ce150-75d7-4032-8e6b-a60c4541599a None admin] RBAC: Authorizing `identity:get_project(project_id=0e2b8dd5a93f4b479d8bc81084d2a51c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:11.491577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ce150-75d7-4032-8e6b-a60c4541599a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:11.493733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 65/136] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:11 2022] GET /identity/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:11.500560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:11.617898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:11.637748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a887d-fdcc-470c-a0c0-3a4daafc419e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:11.659095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf013-a302-4ae5-8eaf-3d7b1986ef51 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:11.659844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf013-a302-4ae5-8eaf-3d7b1986ef51 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:11.660474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf013-a302-4ae5-8eaf-3d7b1986ef51 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:11.662086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 66/137] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:11.667583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c919e6-2c7b-4e61-abc6-37fec963f166 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:11.668134 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c919e6-2c7b-4e61-abc6-37fec963f166 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:11.668671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c919e6-2c7b-4e61-abc6-37fec963f166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:11.688219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:11.688945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:11.689237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:11.689513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] PATH_INFO: `/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c/users/dce6be918e8741208b14eb3247cf7e38/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:11.693284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e2b8dd5a93f4b479d8bc81084d2a51c, user_id=dce6be918e8741208b14eb3247cf7e38, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:11.694766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:11.704646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-05c919e6-2c7b-4e61-abc6-37fec963f166 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:11.706229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05c919e6-2c7b-4e61-abc6-37fec963f166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:11.794720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:11.807977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 72/138] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:11 2022] PUT /identity/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c/users/dce6be918e8741208b14eb3247cf7e38/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:11.809425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 67/139] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:12.362454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:12.489229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812b4b1-becd-4d49-abea-b4354ca3c129 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:12.528802 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffb631-8cce-453b-bad2-0ffb334848f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:12.529454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb631-8cce-453b-bad2-0ffb334848f4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:12.529762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb631-8cce-453b-bad2-0ffb334848f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:12.530050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb631-8cce-453b-bad2-0ffb334848f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:12.531178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffb631-8cce-453b-bad2-0ffb334848f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:12.532608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffb631-8cce-453b-bad2-0ffb334848f4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:12.534449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 73/140] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:12.539548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c919e6-2c7b-4e61-abc6-37fec963f166 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:12.669659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c919e6-2c7b-4e61-abc6-37fec963f166 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:12.712280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abee13-b42b-43ea-aeb1-7b4c974bfa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:12.713583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abee13-b42b-43ea-aeb1-7b4c974bfa6f None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:12.714066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abee13-b42b-43ea-aeb1-7b4c974bfa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:12.714595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abee13-b42b-43ea-aeb1-7b4c974bfa6f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:12.715464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abee13-b42b-43ea-aeb1-7b4c974bfa6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:12.717218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abee13-b42b-43ea-aeb1-7b4c974bfa6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:12.808501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 68/141] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:12 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:55:13.100834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d24d5-36a6-408c-a229-70a9dd6fc37f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:13.101356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d24d5-36a6-408c-a229-70a9dd6fc37f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:13.101356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d24d5-36a6-408c-a229-70a9dd6fc37f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:13.103023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 74/142] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:13.108211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0dfc6-47c3-4a4c-8bd8-5d24c312dd02 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:13.109027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0dfc6-47c3-4a4c-8bd8-5d24c312dd02 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:13.109496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0dfc6-47c3-4a4c-8bd8-5d24c312dd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:13.138609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e9c0dfc6-47c3-4a4c-8bd8-5d24c312dd02 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:13.139835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e9c0dfc6-47c3-4a4c-8bd8-5d24c312dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:13.192630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:13.220580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 69/143] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:13.386145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae358050-0db5-4ed4-8574-d34df3f04fbb None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:13.386419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae358050-0db5-4ed4-8574-d34df3f04fbb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:13.386616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae358050-0db5-4ed4-8574-d34df3f04fbb None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:13.388210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 75/144] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:13.393574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9701c-30f4-4bf3-8ec5-e59660dc0c98 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:13.393874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9701c-30f4-4bf3-8ec5-e59660dc0c98 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:13.394158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9701c-30f4-4bf3-8ec5-e59660dc0c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:13.414866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1ca9701c-30f4-4bf3-8ec5-e59660dc0c98 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:13.416916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ca9701c-30f4-4bf3-8ec5-e59660dc0c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:13.502734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 70/145] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:13.637699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae358050-0db5-4ed4-8574-d34df3f04fbb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:13.661015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:13.770030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae358050-0db5-4ed4-8574-d34df3f04fbb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:13.808910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51e736-f2ad-4ab2-9152-e322d2abf4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:13.809558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51e736-f2ad-4ab2-9152-e322d2abf4ff None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:13.809856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51e736-f2ad-4ab2-9152-e322d2abf4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:13.810125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51e736-f2ad-4ab2-9152-e322d2abf4ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:13.827231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51e736-f2ad-4ab2-9152-e322d2abf4ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2f51e736-f2ad-4ab2-9152-e322d2abf4ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:13.832155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:13.832977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:13.833762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 76/146] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:13.841834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9701c-30f4-4bf3-8ec5-e59660dc0c98 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:13.966120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9701c-30f4-4bf3-8ec5-e59660dc0c98 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:13.975224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5e56-0636-4356-b82c-ca285fb34ca1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:13.975398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5e56-0636-4356-b82c-ca285fb34ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:13.975683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5e56-0636-4356-b82c-ca285fb34ca1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:13.977317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 77/147] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:13.983173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:13.995567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03950ec-4311-4a01-91e5-80f23f7525c6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:13.995840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03950ec-4311-4a01-91e5-80f23f7525c6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:13.996110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03950ec-4311-4a01-91e5-80f23f7525c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.011825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:14.012462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.012748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.013025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.013978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:14.014080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:14.015902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:14.027581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 71/148] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:14.028336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:14.029008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d03950ec-4311-4a01-91e5-80f23f7525c6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:14.034948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d03950ec-4311-4a01-91e5-80f23f7525c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:14.136251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 78/149] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:14.136966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03950ec-4311-4a01-91e5-80f23f7525c6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:14.148894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:14.178810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec527152-52d5-434d-9e0d-c68f946da8be None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:14.228756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4bb9a-a6c7-498c-8f78-014d7d5cfc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:14.229334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4bb9a-a6c7-498c-8f78-014d7d5cfc03 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.230504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4bb9a-a6c7-498c-8f78-014d7d5cfc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.230800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4bb9a-a6c7-498c-8f78-014d7d5cfc03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.251989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4bb9a-a6c7-498c-8f78-014d7d5cfc03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0bf4bb9a-a6c7-498c-8f78-014d7d5cfc03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:14.266770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:14.267681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:14.268430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 72/150] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:14.269831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41fdc3-defe-4b61-9847-a6a3753cad88 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.269831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41fdc3-defe-4b61-9847-a6a3753cad88 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.269831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41fdc3-defe-4b61-9847-a6a3753cad88 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.272462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 73/151] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:14.284758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774feaed-7ed3-4313-80bd-661f72b4954c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.285050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774feaed-7ed3-4313-80bd-661f72b4954c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.285309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774feaed-7ed3-4313-80bd-661f72b4954c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.320501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-774feaed-7ed3-4313-80bd-661f72b4954c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:14.320501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-774feaed-7ed3-4313-80bd-661f72b4954c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:14.331718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03950ec-4311-4a01-91e5-80f23f7525c6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:14.372443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:14.373103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.373671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.373957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.377093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:14.379257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:14.381150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 79/152] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:14 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:14.384766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:14.398739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:14.483062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 74/153] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:14.483681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774feaed-7ed3-4313-80bd-661f72b4954c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:14.506676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:14.535835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251391d-e76a-4889-8b12-5f2e1ed8c764 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:14.579095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:14.579723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.580005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.580281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.581071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:14.581383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:14.582789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:14.593137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 80/154] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:14.593859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:14.690514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774feaed-7ed3-4313-80bd-661f72b4954c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:14.733188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afcb98-6fb7-4551-9ea9-f1bb57561429 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:14.741191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad798-754c-437f-99e2-194b042a979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:14.742628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad798-754c-437f-99e2-194b042a979a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.743120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad798-754c-437f-99e2-194b042a979a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.744563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad798-754c-437f-99e2-194b042a979a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.789936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443656b-2220-47da-a8c3-de46a091ed44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:14.792070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443656b-2220-47da-a8c3-de46a091ed44 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.792674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443656b-2220-47da-a8c3-de46a091ed44 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.793290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443656b-2220-47da-a8c3-de46a091ed44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.799561 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:14.808310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:14.820678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443656b-2220-47da-a8c3-de46a091ed44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3443656b-2220-47da-a8c3-de46a091ed44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:14.826720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:14.827657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:14.828413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:14.834849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 81/155] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:14.836694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb1f2e-4296-4d35-8e5c-d31b07663f05 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:14.836981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb1f2e-4296-4d35-8e5c-d31b07663f05 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:14.837257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb1f2e-4296-4d35-8e5c-d31b07663f05 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:14.840569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 82/156] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:14.841246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb1f2e-4296-4d35-8e5c-d31b07663f05 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:14.893392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad798-754c-437f-99e2-194b042a979a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:14.895069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad798-754c-437f-99e2-194b042a979a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:15.048261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb1f2e-4296-4d35-8e5c-d31b07663f05 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:15.087966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 75/157] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:15.089345 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296461-ea7f-47e7-8225-51f2ab279c24 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:15.093780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296461-ea7f-47e7-8225-51f2ab279c24 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:15.094093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296461-ea7f-47e7-8225-51f2ab279c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:15.118194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:15.118974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:15.119255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:15.119533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] PATH_INFO: `/v3/users/f100ee5a590a4764a1d39fd503354ae4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:15.122237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] RBAC: Authorizing `identity:get_user(user_id=f100ee5a590a4764a1d39fd503354ae4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:15.123702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:15.125466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 83/158] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:14 2022] GET /identity/v3/users/f100ee5a590a4764a1d39fd503354ae4 => generated 247 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:15.126436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:15.133605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ed296461-ea7f-47e7-8225-51f2ab279c24 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:15.135732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed296461-ea7f-47e7-8225-51f2ab279c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:15.211027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:15.253531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10310e08-e701-40d2-877c-8d7e87892e19 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:15.287299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:15.293019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:15.293665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:15.293954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:15.294235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:15.294906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:15.295214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:15.296608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:15.304692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 76/159] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:15.309111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296461-ea7f-47e7-8225-51f2ab279c24 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:15.335890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 84/160] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:15.336784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:15.490454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296461-ea7f-47e7-8225-51f2ab279c24 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:15.557003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:15.558083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:15.558585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:15.559056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:15.582069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:15.590777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:15.591690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:15.592483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:15.600481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 77/161] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:15.601515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:15.625298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2aa8-49b2-4f31-afdf-da7b87f88b1e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:15.674064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:15.690264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:15.690917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:15.691280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:15.708419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:15.708419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:15.708419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:15.708419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 85/162] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:15 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:15.708419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:15.708785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:15.807942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1920a95-c4b6-42c3-ba8d-725720f73f91 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:15.837047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6128-b6b0-425e-9fda-ea69c9d64e09 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:15.880564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:15.881424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:15.882031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:15.883620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:15.910928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:15.911420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:15.911698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:15.911978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] PATH_INFO: `/v3/users/263efbc979c844daa8077f6274bc6521` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:15.918424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] RBAC: Authorizing `identity:get_user(user_id=263efbc979c844daa8077f6274bc6521)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:15.919903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:15.921612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 86/163] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:15 2022] GET /identity/v3/users/263efbc979c844daa8077f6274bc6521 => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:15.922384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.007754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:16.010754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.012480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.081238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367bcdc-fa36-44df-b7a2-0d9c029c561d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.135900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.136774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.137205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.137794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.138728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:16.140908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.142491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.142959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:16.167300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 87/164] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:16.167300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.217092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 78/165] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:16.218070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.294978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72a693-7d9b-4b10-9001-43272c3f61e7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.319911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:16.330584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc724-beba-4ee1-8864-674af0b49012 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.370116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.370116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.370116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.370116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.371159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=300553fe41194f45839b5a1043768774, user.id=dce6be918e8741208b14eb3247cf7e38) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:16.371159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.371159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.371159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 88/166] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:16 2022] GET /identity/v3/role_assignments?user.id=dce6be918e8741208b14eb3247cf7e38&scope.project.id=300553fe41194f45839b5a1043768774&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:16.371159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.371524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1d684-045d-4b25-b35b-12fc410a1766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.371524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1d684-045d-4b25-b35b-12fc410a1766 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.371524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1d684-045d-4b25-b35b-12fc410a1766 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.371524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1d684-045d-4b25-b35b-12fc410a1766 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.380244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1d684-045d-4b25-b35b-12fc410a1766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a8b1d684-045d-4b25-b35b-12fc410a1766 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:16.386139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:16.387532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:16.388566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 79/167] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:16.389281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a7ead-1ed4-4cf7-ba05-2916ac5ffcce None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.389281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a7ead-1ed4-4cf7-ba05-2916ac5ffcce None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.389281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a7ead-1ed4-4cf7-ba05-2916ac5ffcce None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.394141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 80/168] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:16.395025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a7ead-1ed4-4cf7-ba05-2916ac5ffcce None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.413600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:16.466428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb971a-380b-4dcd-814b-ba84d9afb19c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.504426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.505368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.505834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.506263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.507800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.508057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a7ead-1ed4-4cf7-ba05-2916ac5ffcce None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.510235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.512357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 89/169] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:16 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:16.513283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.557454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.559123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.560169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.560459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.563376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.566339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.570506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 81/170] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:16 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:16.572513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.620259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:16.685802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:16.706714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef726810-d8ff-4f34-ac18-fb0ef097b06f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.724134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053afd3-9dc6-498b-b952-9f2a57f80bb3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.758190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.760129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.760718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.761470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.763014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:16.763628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.765307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.767028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983642a-f0c1-4084-9b88-8c4c7084899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.767791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983642a-f0c1-4084-9b88-8c4c7084899b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.769046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983642a-f0c1-4084-9b88-8c4c7084899b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.769425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983642a-f0c1-4084-9b88-8c4c7084899b None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.770750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983642a-f0c1-4084-9b88-8c4c7084899b None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.772396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983642a-f0c1-4084-9b88-8c4c7084899b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.774062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 90/171] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:16 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:16.775915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463e9f3-8b7d-4941-9263-d33ad9e67c64 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.776301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463e9f3-8b7d-4941-9263-d33ad9e67c64 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.776679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463e9f3-8b7d-4941-9263-d33ad9e67c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.777070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 82/172] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:16.778072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.807434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4463e9f3-8b7d-4941-9263-d33ad9e67c64 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:16.809304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4463e9f3-8b7d-4941-9263-d33ad9e67c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:16.891424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 91/173] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:16.892362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463e9f3-8b7d-4941-9263-d33ad9e67c64 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:16.913920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0865081-4239-4320-916e-8f1f87eb1602 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:16.951683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:16.961941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:16.963613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:16.964067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:16.965003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774/users/263efbc979c844daa8077f6274bc6521/roles/7b93900f6d71493ca4954f9eba5d159b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:16.967562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=300553fe41194f45839b5a1043768774, user_id=263efbc979c844daa8077f6274bc6521, role_id=7b93900f6d71493ca4954f9eba5d159b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:16.970068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:16.999856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463e9f3-8b7d-4941-9263-d33ad9e67c64 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:17.036817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e21d6-f460-4990-a382-b914ce9f574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:17.037672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e21d6-f460-4990-a382-b914ce9f574c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.037999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e21d6-f460-4990-a382-b914ce9f574c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.038274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e21d6-f460-4990-a382-b914ce9f574c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.045667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 83/174] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:16 2022] PUT /identity/v3/projects/300553fe41194f45839b5a1043768774/users/263efbc979c844daa8077f6274bc6521/roles/7b93900f6d71493ca4954f9eba5d159b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:17.057820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:17.077958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:17.155125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e21d6-f460-4990-a382-b914ce9f574c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:17.156515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e21d6-f460-4990-a382-b914ce9f574c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:17.195293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba62a-2834-4f3c-aab2-3300718032d8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:17.217553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:17.237079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:17.237842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.238124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.238398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] PATH_INFO: `/v3/users/dce6be918e8741208b14eb3247cf7e38` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.239587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] RBAC: Authorizing `identity:get_user(user_id=dce6be918e8741208b14eb3247cf7e38)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:17.241155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:17.242985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 84/175] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:17 2022] GET /identity/v3/users/dce6be918e8741208b14eb3247cf7e38 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:17.243625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:17.276450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 92/176] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:17.278980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf78c-3ed9-4795-93f6-6408c385d3c2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.279440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf78c-3ed9-4795-93f6-6408c385d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.279860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf78c-3ed9-4795-93f6-6408c385d3c2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.282479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 93/177] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:17.286452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf67ec-fd0e-484b-99a4-697ba9d2dfb4 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.286452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf67ec-fd0e-484b-99a4-697ba9d2dfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.286452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf67ec-fd0e-484b-99a4-697ba9d2dfb4 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.287247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 94/178] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:17.287863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf67ec-fd0e-484b-99a4-697ba9d2dfb4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:17.325652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:17.357835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaedd50-795d-4cd2-9ade-cd8e98bbfed7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:17.396310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:17.396933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.397215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.397491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.398388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=300553fe41194f45839b5a1043768774, user.id=f100ee5a590a4764a1d39fd503354ae4) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:17.398698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:17.400359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:17.401791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf67ec-fd0e-484b-99a4-697ba9d2dfb4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:17.413666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 85/179] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:17 2022] GET /identity/v3/role_assignments?user.id=f100ee5a590a4764a1d39fd503354ae4&scope.project.id=300553fe41194f45839b5a1043768774&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:17.413666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:17.444292 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:17.445040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.445394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.445787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.457992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:17.463796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:17.464723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:17.465744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 95/180] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:17.468385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:17.490513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:17.556020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c1dae-ae06-49d4-ab06-a600df9fb1cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:17.591734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b905b-a6f6-4729-8f10-d2037470d9a6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:17.607113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:17.608055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.608528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.609010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.612999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:17.635038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5cc7c-17d4-4e3a-9cb5-b055f3d5a011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:17.636952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5cc7c-17d4-4e3a-9cb5-b055f3d5a011 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.637304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5cc7c-17d4-4e3a-9cb5-b055f3d5a011 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.637711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5cc7c-17d4-4e3a-9cb5-b055f3d5a011 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.638951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5cc7c-17d4-4e3a-9cb5-b055f3d5a011 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:17.640467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5cc7c-17d4-4e3a-9cb5-b055f3d5a011 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:17.642348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 96/181] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:17 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:17.643359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93af12a-43e2-441a-b04d-c42993c08a3f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.643641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93af12a-43e2-441a-b04d-c42993c08a3f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.643920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93af12a-43e2-441a-b04d-c42993c08a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.663123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b93af12a-43e2-441a-b04d-c42993c08a3f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:17.664889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b93af12a-43e2-441a-b04d-c42993c08a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:17.696370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:17.700067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:17.746451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 97/182] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:17.747669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd293a8-ea72-4704-9265-1782887ed454 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:17.748109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd293a8-ea72-4704-9265-1782887ed454 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:17.748490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd293a8-ea72-4704-9265-1782887ed454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:17.767400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4dd293a8-ea72-4704-9265-1782887ed454 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:17.768660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4dd293a8-ea72-4704-9265-1782887ed454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:17.779086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:17.820490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 86/183] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:17.822323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:17.862064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 98/184] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:17.863339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd293a8-ea72-4704-9265-1782887ed454 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:17.867666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.000389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53bb8-c269-49a4-9dd7-e3ff84820e41 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.050427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd293a8-ea72-4704-9265-1782887ed454 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.086236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.086857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.087139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.088058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.088786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:18.089095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.090528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.100166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 87/185] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:18.100469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.101095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.101378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.101685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.102206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.102421 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] RBAC: Adding query filter params (role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=300553fe41194f45839b5a1043768774, user.id=dce6be918e8741208b14eb3247cf7e38) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:18.102734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.108891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.115744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 99/186] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:17 2022] GET /identity/v3/role_assignments?user.id=dce6be918e8741208b14eb3247cf7e38&scope.project.id=300553fe41194f45839b5a1043768774&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:18.121762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.132261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.269907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18428e11-345b-4c16-b00a-a3ec9e46c5e0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.290629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.294445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713472c-2943-4ea3-9ec1-02f6167a2891 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.314993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.315641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.315927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.316206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.331106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:18.338570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:18.339450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:18.340229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 100/187] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:18.340891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.346496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.347256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.347639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.348017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] PATH_INFO: `/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.349323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=0e2b8dd5a93f4b479d8bc81084d2a51c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.350951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.352798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 88/188] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:18 2022] GET /identity/v3/projects/0e2b8dd5a93f4b479d8bc81084d2a51c => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:18.353692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.424886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.463586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a80ed2-169b-4890-98d0-4e1b4423bcaa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.488022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72508f5e-c0a2-4f15-a842-9bb17d563bb6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.500366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.500973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.501517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.501853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] PATH_INFO: `/v3/users/3d2c3e1c822840b3beac54677801f94a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.517325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] RBAC: Authorizing `identity:get_user(user_id=3d2c3e1c822840b3beac54677801f94a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.518849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.520552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 101/189] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:18 2022] GET /identity/v3/users/3d2c3e1c822840b3beac54677801f94a => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:18.524252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.525775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.525892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.526107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.526463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.527690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.529151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.530807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 89/190] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:18.530807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.572228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.636060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27249882-2510-4c91-a05b-d332d0bb475f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.645596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256881-a382-473c-99e1-f6823197fa33 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.661673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.672475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.673084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.673362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.673664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.674329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:18.674630 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.676118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.682734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.683356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.683641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.683918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.684560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 102/191] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:18.684667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=0e2b8dd5a93f4b479d8bc81084d2a51c, user.id=dce6be918e8741208b14eb3247cf7e38) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:18.684949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.685166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.686504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.692727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 90/192] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:18 2022] GET /identity/v3/role_assignments?user.id=dce6be918e8741208b14eb3247cf7e38&scope.project.id=0e2b8dd5a93f4b479d8bc81084d2a51c&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:18.693364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:18.827967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c4e21-4e02-4642-aa74-299dfc051099 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.847747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.852727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f16346-ff0f-4430-be81-5796c74bd569 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:18.876725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.877355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.877656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.878007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.895823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:18.897455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:18.897455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:18.897455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:18.897739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.899904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:18.945628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:18.958909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:18.960408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:19.007794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 91/193] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:18 2022] POST /identity/v3/users => generated 297 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:19.008740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:19.083031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 103/194] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:19.089180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:19.146175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b970afc-e2d5-418d-8d1e-a881ea318b6f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:19.188238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:19.194216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cad98-5e08-4b00-b6e9-d080991e4cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:19.195178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cad98-5e08-4b00-b6e9-d080991e4cac None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.195559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cad98-5e08-4b00-b6e9-d080991e4cac None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.195831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cad98-5e08-4b00-b6e9-d080991e4cac None admin] PATH_INFO: `/v3/users/3d2c3e1c822840b3beac54677801f94a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.197650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cad98-5e08-4b00-b6e9-d080991e4cac None admin] RBAC: Authorizing `identity:get_user(user_id=3d2c3e1c822840b3beac54677801f94a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:19.199866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cad98-5e08-4b00-b6e9-d080991e4cac None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:19.200832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 92/195] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:19 2022] GET /identity/v3/users/3d2c3e1c822840b3beac54677801f94a => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:19.202132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff1b7-5946-4158-ad1c-9668097b9922 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.202441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff1b7-5946-4158-ad1c-9668097b9922 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.202719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff1b7-5946-4158-ad1c-9668097b9922 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.204051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 93/196] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:19.204768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86763269-e9c1-4eb3-be5c-896221fa3cdc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:19.206425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bff1b7-5946-4158-ad1c-9668097b9922 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:19.235132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:19.245062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc654-fbe9-42d4-9f91-1fe5fd6c0a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:19.245743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc654-fbe9-42d4-9f91-1fe5fd6c0a37 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.246489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc654-fbe9-42d4-9f91-1fe5fd6c0a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.246489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc654-fbe9-42d4-9f91-1fe5fd6c0a37 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.247490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dc654-fbe9-42d4-9f91-1fe5fd6c0a37 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:19.248933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dc654-fbe9-42d4-9f91-1fe5fd6c0a37 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:19.250674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 104/197] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:19 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:19.253015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee775731-9832-4e61-bcfb-7fd7ef334281 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.253015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee775731-9832-4e61-bcfb-7fd7ef334281 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.253015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee775731-9832-4e61-bcfb-7fd7ef334281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.271048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ee775731-9832-4e61-bcfb-7fd7ef334281 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:19.272580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ee775731-9832-4e61-bcfb-7fd7ef334281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:19.354563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bff1b7-5946-4158-ad1c-9668097b9922 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:19.371748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 105/198] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:19.372608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee775731-9832-4e61-bcfb-7fd7ef334281 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:19.397294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:19.398257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.398911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.399376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.474024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:19.475911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:19.481594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:19.505909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:19.511663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee775731-9832-4e61-bcfb-7fd7ef334281 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:19.588447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:19.588941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.589197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.589469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.590314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=3d2c3e1c822840b3beac54677801f94a) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:19.590623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:19.592085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:19.603567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 106/199] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:19 2022] GET /identity/v3/role_assignments?user.id=3d2c3e1c822840b3beac54677801f94a&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:19.620273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 94/200] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:55:19.627001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:19.765886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160b86-7660-4eeb-9b20-811c76993d72 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:19.802135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:19.802782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.803061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.803334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.804478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:19.806474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:19.808511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 107/201] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:19 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:19.814625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:19.828422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:19.851799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:19.939383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd4371-08e6-4d08-9a9f-495bb01ea024 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:19.947373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94083a9-1409-4e88-b584-4a3d0f21de31 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:19.972173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:19.975451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:19.976098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.976382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.976649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.986220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:19.987127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:19.987600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:19.987787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:19.988263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:19.989191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] RBAC: Adding query filter params (role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=3d2c3e1c822840b3beac54677801f94a) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:19.989721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:19.991377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:19.992729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:19.994430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:19.995219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 108/202] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:20.001990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 95/203] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:19 2022] GET /identity/v3/role_assignments?user.id=3d2c3e1c822840b3beac54677801f94a&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:20.006693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:20.131467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:20.158382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3cff-476b-4663-b12a-6a91aff6406e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:20.195585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:20.196197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:20.196478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:20.197414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:20.198120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:20.198423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:20.199793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:20.208243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 109/204] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:20.214284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:20.364333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23bf7-14dc-4aa8-a786-9ee007dc722f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:20.400220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:20.400788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:20.401054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:20.401327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] PATH_INFO: `/v3/users/263efbc979c844daa8077f6274bc6521` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:20.402492 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] RBAC: Authorizing `identity:get_user(user_id=263efbc979c844daa8077f6274bc6521)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:20.403949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:20.405984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 96/205] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:20 2022] GET /identity/v3/users/263efbc979c844daa8077f6274bc6521 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:20.411405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:20.535874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564968-c247-4b30-8f88-c6e2fff7110d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:20.581966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:20.582582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:20.582863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:20.583174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:20.592975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:20.672578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:20.673991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:20.806580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 110/206] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:55:20.812284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:20.880314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:20.927303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc02ca-21f7-469b-b0c2-0b2438a56bdb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:20.970864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4656b1-5efa-40d4-892c-29649c4b59b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:20.971510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4656b1-5efa-40d4-892c-29649c4b59b8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:20.971796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4656b1-5efa-40d4-892c-29649c4b59b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:20.972076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4656b1-5efa-40d4-892c-29649c4b59b8 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:20.973222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4656b1-5efa-40d4-892c-29649c4b59b8 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:20.979402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4656b1-5efa-40d4-892c-29649c4b59b8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:20.981475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 97/207] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:20 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:20.987549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:21.035513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:21.114041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058a9f-1477-4c1f-913e-b6e6217c630e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:21.161216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d265c2e-fc8a-4bed-a00a-582a71fdb049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:21.161870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d265c2e-fc8a-4bed-a00a-582a71fdb049 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:21.162150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d265c2e-fc8a-4bed-a00a-582a71fdb049 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:21.162421 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d265c2e-fc8a-4bed-a00a-582a71fdb049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:21.163174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d265c2e-fc8a-4bed-a00a-582a71fdb049 None admin] RBAC: Adding query filter params (role.id=7b93900f6d71493ca4954f9eba5d159b, scope.project.id=300553fe41194f45839b5a1043768774, user.id=263efbc979c844daa8077f6274bc6521) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:21.163486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d265c2e-fc8a-4bed-a00a-582a71fdb049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:21.164968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d265c2e-fc8a-4bed-a00a-582a71fdb049 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:21.175023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 111/208] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:20 2022] GET /identity/v3/role_assignments?user.id=263efbc979c844daa8077f6274bc6521&scope.project.id=300553fe41194f45839b5a1043768774&role.id=7b93900f6d71493ca4954f9eba5d159b => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:21.441145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694593f-c264-465d-9f0d-8862586e0cab None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:21.441468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694593f-c264-465d-9f0d-8862586e0cab None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:21.441792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694593f-c264-465d-9f0d-8862586e0cab None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:21.443165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 98/209] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:21.448640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71543491-5bf8-48e1-b787-456c51ff06a4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:21.448932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71543491-5bf8-48e1-b787-456c51ff06a4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:21.449209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71543491-5bf8-48e1-b787-456c51ff06a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:21.468521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-71543491-5bf8-48e1-b787-456c51ff06a4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:21.469756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-71543491-5bf8-48e1-b787-456c51ff06a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:21.548259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 112/210] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:21.832919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694593f-c264-465d-9f0d-8862586e0cab None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:21.938172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694593f-c264-465d-9f0d-8862586e0cab None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:21.960646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:21.972619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dedab9-9ea6-4a05-84fe-2c22dc99a403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:21.979330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dedab9-9ea6-4a05-84fe-2c22dc99a403 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:21.979330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dedab9-9ea6-4a05-84fe-2c22dc99a403 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:21.979330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dedab9-9ea6-4a05-84fe-2c22dc99a403 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:21.979330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dedab9-9ea6-4a05-84fe-2c22dc99a403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:21.979330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dedab9-9ea6-4a05-84fe-2c22dc99a403 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:21.979330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 99/211] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:21.983867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71543491-5bf8-48e1-b787-456c51ff06a4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:22.016995 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:22.028712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd621-cf89-4418-9fd5-07506a987ea8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.029009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd621-cf89-4418-9fd5-07506a987ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.033894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd621-cf89-4418-9fd5-07506a987ea8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.033894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 100/212] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:22.038306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add618fd-63c5-43d5-a628-85a8cee217f9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.038594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add618fd-63c5-43d5-a628-85a8cee217f9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.038876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add618fd-63c5-43d5-a628-85a8cee217f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.062030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-add618fd-63c5-43d5-a628-85a8cee217f9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:22.063481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-add618fd-63c5-43d5-a628-85a8cee217f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:22.101144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71543491-5bf8-48e1-b787-456c51ff06a4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:22.137245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 101/213] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:22.138568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e590cc-7746-4e1b-8634-7c028ae21add None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.138880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e590cc-7746-4e1b-8634-7c028ae21add None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.139154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e590cc-7746-4e1b-8634-7c028ae21add None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.140582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 102/214] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:22.145458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba295be1-f54c-4c2c-9a9f-c571ab6a4369 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.145887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba295be1-f54c-4c2c-9a9f-c571ab6a4369 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.146098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba295be1-f54c-4c2c-9a9f-c571ab6a4369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.159751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:22.161067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.161627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.162125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.163023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:22.164310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ba295be1-f54c-4c2c-9a9f-c571ab6a4369 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:22.165095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:22.165544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba295be1-f54c-4c2c-9a9f-c571ab6a4369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:22.235824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 103/215] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:22.243032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a04a-4c38-44f2-9cec-4323722969b6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.243319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a04a-4c38-44f2-9cec-4323722969b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.243670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a04a-4c38-44f2-9cec-4323722969b6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.245610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 104/216] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:22.250124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cafa6c-2e5e-4879-bb0e-27c65b56a0e6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.250408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cafa6c-2e5e-4879-bb0e-27c65b56a0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.250681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cafa6c-2e5e-4879-bb0e-27c65b56a0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.269476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-57cafa6c-2e5e-4879-bb0e-27c65b56a0e6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:22.273180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-57cafa6c-2e5e-4879-bb0e-27c65b56a0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:22.278385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 113/217] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:21 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:22.341308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:22.366740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 105/218] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:22.418390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:22.515263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:22.518208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cafa6c-2e5e-4879-bb0e-27c65b56a0e6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:22.533815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb8bb4-0079-4e1f-9439-ac3b26b2be95 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:22.583120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:22.583120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.583120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.583120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.594618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:22.599582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:22.600518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:22.601280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:22.603347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 114/219] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:22.608287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:22.674858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cafa6c-2e5e-4879-bb0e-27c65b56a0e6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:22.730025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:22.730688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.730972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.731272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.741154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aac1d1-5dc2-43b6-86b8-46b0d8f0ac1d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:22.746746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:22.752633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:22.753519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:22.754342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 106/220] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:22.754991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:22.784822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:22.785862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.786487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.786961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.787871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:22.788376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:22.790679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:22.800543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 115/221] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:22.801848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:22.814078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:22.825628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:22.883724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1168-ff78-44fd-9029-bd11279ee65a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:22.928318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f1cd-8f7f-442b-a0e0-162ff7b92cb1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:22.928970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:22.929643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.929929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.930205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.944650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:22.950155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:22.951019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:22.951803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:22.954507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 107/222] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:22.955542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:22.973000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:22.974108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:22.974595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:22.975120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:22.976037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:22.979085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:22.980855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:22.994486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 116/223] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:22.995883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.073919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.075645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.077818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ef2aa-1bd5-47e8-abbb-c1a3d8e4d6a4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.108329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620f780-b266-4ff8-8c5c-a7849b9cf28c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.113982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.114819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.115246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.115666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] PATH_INFO: `/v3/users/dfdd91f3e0784e2f95f67e4eb703842e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.130067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] RBAC: Authorizing `identity:get_user(user_id=dfdd91f3e0784e2f95f67e4eb703842e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.131791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.139675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 108/224] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:22 2022] GET /identity/v3/users/dfdd91f3e0784e2f95f67e4eb703842e => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:23.141100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.155677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.156536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.156955 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.157386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.158294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:23.158754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.160276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.169854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 117/225] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:23.170926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.268433 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e88a-a8b8-440f-964e-513daa00dcb7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.306660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.307774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.308333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.308814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] PATH_INFO: `/v3/users/3d2c3e1c822840b3beac54677801f94a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.310179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] RBAC: Authorizing `identity:get_user(user_id=3d2c3e1c822840b3beac54677801f94a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.311026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84670a9a-feed-48bc-b04c-bcfaebbdd6eb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.312211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.314250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 109/226] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:23 2022] GET /identity/v3/users/3d2c3e1c822840b3beac54677801f94a => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:23.315264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.351779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.358779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.359409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.359730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.360011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.363618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.441617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.443624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.450107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992bd5-46cf-4d44-9445-d6cc40f5f023 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.490516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.491181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.491526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.491800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] PATH_INFO: `/v3/users/3d2c3e1c822840b3beac54677801f94a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.492977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=3d2c3e1c822840b3beac54677801f94a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.494509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.496094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 110/227] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:23 2022] GET /identity/v3/users/3d2c3e1c822840b3beac54677801f94a => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:23.497035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.585132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 118/228] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:23.586035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.613604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.620908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f110f-0433-40dd-be16-97401cf25fc6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.637709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.657447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.657884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.658400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.658745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.660004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.661502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.664756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 111/229] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:23 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:23.664756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.729101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec98e3-de88-4c39-a77b-b9a7358d4acf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.773058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4aa38-64fb-4a47-af6d-38eca8b8617d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.806675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.807520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.807808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.808180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.809622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.809917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.810616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.810958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.811065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.811335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.812514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.814195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.815893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 112/231] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:23 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:23.816714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.817928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 119/231] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:23 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:23.818745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:23.850412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.931759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2e1a1-6251-4331-8be6-8ce16ab24be3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.954582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc53598-0cd9-40fd-b1b4-388ea550e733 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:23.956819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:23.969187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.969979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.970379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:23.970743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/3d2c3e1c822840b3beac54677801f94a/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:23.972783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c320d38a1546d0a6784ac68419376d, user_id=3d2c3e1c822840b3beac54677801f94a, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:23.974410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:23.998197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856eafd-753b-49de-9ee9-05abd8b151c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:23.999147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856eafd-753b-49de-9ee9-05abd8b151c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:23.999601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856eafd-753b-49de-9ee9-05abd8b151c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:24.000029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856eafd-753b-49de-9ee9-05abd8b151c2 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/3d2c3e1c822840b3beac54677801f94a/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:24.002668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856eafd-753b-49de-9ee9-05abd8b151c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c320d38a1546d0a6784ac68419376d, user_id=3d2c3e1c822840b3beac54677801f94a, role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:24.004272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856eafd-753b-49de-9ee9-05abd8b151c2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:24.038219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 113/232] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:23 2022] PUT /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/3d2c3e1c822840b3beac54677801f94a/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:24.039182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:24.063365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 120/233] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:23 2022] PUT /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/3d2c3e1c822840b3beac54677801f94a/roles/eb0cba906bfe429883619da6e985a2b4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:24.194339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:24.215879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b0f5-6eff-469c-b254-ba3e54c70a2c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:24.271232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57ae09-4ce9-465b-af04-a7b5d7d9454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:24.272104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ae09-4ce9-465b-af04-a7b5d7d9454f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:24.272402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ae09-4ce9-465b-af04-a7b5d7d9454f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:24.272748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ae09-4ce9-465b-af04-a7b5d7d9454f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:24.273499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57ae09-4ce9-465b-af04-a7b5d7d9454f None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=dfdd91f3e0784e2f95f67e4eb703842e) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:24.273833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57ae09-4ce9-465b-af04-a7b5d7d9454f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:24.276087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57ae09-4ce9-465b-af04-a7b5d7d9454f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:24.282719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 114/234] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:24 2022] GET /identity/v3/role_assignments?user.id=dfdd91f3e0784e2f95f67e4eb703842e&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:24.555887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d67fd0-0f19-459e-9a6a-9da2dd4ce114 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:24.555887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d67fd0-0f19-459e-9a6a-9da2dd4ce114 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:24.555887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d67fd0-0f19-459e-9a6a-9da2dd4ce114 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:24.555887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 121/235] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:24.564497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95de3af-4881-426b-8ac0-2ffc50ad10a5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:24.564785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95de3af-4881-426b-8ac0-2ffc50ad10a5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:24.565057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95de3af-4881-426b-8ac0-2ffc50ad10a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:24.594710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b95de3af-4881-426b-8ac0-2ffc50ad10a5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:24.595934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b95de3af-4881-426b-8ac0-2ffc50ad10a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:24.707557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26dea4-a45e-46cd-ad1b-40a61451a3b8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:24.708138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26dea4-a45e-46cd-ad1b-40a61451a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:24.708138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26dea4-a45e-46cd-ad1b-40a61451a3b8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:24.709599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 122/236] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:24.715981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a0e0-5eb1-43f1-9525-9f46f89e6ae5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:24.715981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a0e0-5eb1-43f1-9525-9f46f89e6ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:24.716655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a0e0-5eb1-43f1-9525-9f46f89e6ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:24.725587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 115/237] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:24.768667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d627a0e0-5eb1-43f1-9525-9f46f89e6ae5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:24.770004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d627a0e0-5eb1-43f1-9525-9f46f89e6ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:24.906941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 123/238] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:25.046631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95de3af-4881-426b-8ac0-2ffc50ad10a5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:25.137558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:25.153291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95de3af-4881-426b-8ac0-2ffc50ad10a5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:25.197542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:25.198131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:25.198410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:25.198682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:25.213159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:25.218106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:25.218973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:25.219760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 116/239] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:25.224936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a0e0-5eb1-43f1-9525-9f46f89e6ae5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:25.240822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:25.350163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:25.356919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a0e0-5eb1-43f1-9525-9f46f89e6ae5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:25.397328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5857-f3ea-4004-ba85-7c221daf21b6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:25.421599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:25.422228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:25.422511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:25.422785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:25.423459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:25.423764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:25.425145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:25.442894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 124/240] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:25.452927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:25.463736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:25.464393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:25.464672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:25.464945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:25.468353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:25.469871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:25.471780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 117/241] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:25.476886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:25.497375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:25.648759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad07a9-7859-45df-bd0f-bce5b88461a8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:25.654502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd4177-304f-4197-8f7b-63bf89b46a45 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:25.718310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:25.718852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:25.719131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:25.719406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:25.720059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:25.721511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:25.725003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:25.725750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:25.726031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:25.737677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] PATH_INFO: `/v3/users/17c9be0c85e842a2b70ebf89e095f7c2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:25.749732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] RBAC: Authorizing `identity:get_user(user_id=17c9be0c85e842a2b70ebf89e095f7c2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:25.751460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:25.753128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 125/242] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:25 2022] GET /identity/v3/users/17c9be0c85e842a2b70ebf89e095f7c2 => generated 297 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:25.762458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:25.801927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:25.845008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 118/243] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:25 2022] POST /identity/v3/users => generated 293 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:55:25.879169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4576de-3041-46f5-a6db-8530247f3072 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:25.917148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:25.917860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:25.918139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:25.918408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:25.991989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:25.993993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:26.009048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:26.100004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 126/244] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:26.106433 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:26.183136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:26.220201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cddd0-31d5-46f9-b6b0-10577a8e220c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:26.258015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7831d-0901-454d-814e-79e9dff2b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:26.258578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7831d-0901-454d-814e-79e9dff2b8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.259432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7831d-0901-454d-814e-79e9dff2b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.259711 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7831d-0901-454d-814e-79e9dff2b8c3 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.260906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7831d-0901-454d-814e-79e9dff2b8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:26.262426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7831d-0901-454d-814e-79e9dff2b8c3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:26.264065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 119/245] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:26 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:26.270103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:26.393380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb745b-679e-4551-93c6-7edbf67c586e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:26.412128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:26.445369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda1ada-9335-4eb1-8cea-a6e4a6ec31f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:26.446009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda1ada-9335-4eb1-8cea-a6e4a6ec31f6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.446290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda1ada-9335-4eb1-8cea-a6e4a6ec31f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.446655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda1ada-9335-4eb1-8cea-a6e4a6ec31f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.447493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda1ada-9335-4eb1-8cea-a6e4a6ec31f6 None admin] RBAC: Adding query filter params (role.id=7b93900f6d71493ca4954f9eba5d159b, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=17c9be0c85e842a2b70ebf89e095f7c2) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:26.447839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda1ada-9335-4eb1-8cea-a6e4a6ec31f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:26.449319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda1ada-9335-4eb1-8cea-a6e4a6ec31f6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:26.455615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 127/246] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:26 2022] GET /identity/v3/role_assignments?user.id=17c9be0c85e842a2b70ebf89e095f7c2&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=7b93900f6d71493ca4954f9eba5d159b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:26.578605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed141-590a-4d5b-a0a4-7d92fdc27353 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.578940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed141-590a-4d5b-a0a4-7d92fdc27353 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.579115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed141-590a-4d5b-a0a4-7d92fdc27353 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.580724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 120/247] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:26.585969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7552-1943-465c-a20d-2324857aaa9f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.586310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7552-1943-465c-a20d-2324857aaa9f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.586586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7552-1943-465c-a20d-2324857aaa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.612338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0d4c7552-1943-465c-a20d-2324857aaa9f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:26.613545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0d4c7552-1943-465c-a20d-2324857aaa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:26.642661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964158ee-63a3-4271-aaf8-ad052e2bd18a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.642948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964158ee-63a3-4271-aaf8-ad052e2bd18a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.643217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964158ee-63a3-4271-aaf8-ad052e2bd18a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.644565 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 121/248] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:26.650596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c135d-0e9e-4658-850c-cedfdfa2a7ef None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.650880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c135d-0e9e-4658-850c-cedfdfa2a7ef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.651153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c135d-0e9e-4658-850c-cedfdfa2a7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.676348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cc5c135d-0e9e-4658-850c-cedfdfa2a7ef None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:26.677554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cc5c135d-0e9e-4658-850c-cedfdfa2a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:26.698006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 128/249] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:26.706519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46b666-4b9a-4c93-93a2-43c4674a7ca6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.706809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46b666-4b9a-4c93-93a2-43c4674a7ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.707094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46b666-4b9a-4c93-93a2-43c4674a7ca6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.708434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 129/250] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:26.713747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae8935-cebb-4542-a388-903ffad189c3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:26.714030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae8935-cebb-4542-a388-903ffad189c3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:26.714305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae8935-cebb-4542-a388-903ffad189c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:26.742284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a6ae8935-cebb-4542-a388-903ffad189c3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:26.745232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a6ae8935-cebb-4542-a388-903ffad189c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:26.781591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 122/251] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:26.830277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 130/252] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:27.107579 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c135d-0e9e-4658-850c-cedfdfa2a7ef None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:27.177849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:27.214770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c135d-0e9e-4658-850c-cedfdfa2a7ef None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:27.241789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae8935-cebb-4542-a388-903ffad189c3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:27.250386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:27.251305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:27.251591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:27.251867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:27.265605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:27.268097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:27.268954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:27.269763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 123/253] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:27.275929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:27.332580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:27.409875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9d35-be5e-469e-9ac4-657a62d4f218 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:27.431909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae8935-cebb-4542-a388-903ffad189c3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:27.443053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:27.454102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:27.454733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:27.455456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:27.455738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:27.456440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:27.456998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:27.458574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:27.469211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 124/254] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:27.470351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:27.501709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:27.502355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:27.502642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:27.502915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:27.525441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:27.536079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:27.536940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:27.537776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:27.548407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 131/255] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:27.552750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:27.652240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91779e7b-5ab6-4b8b-976a-c3d217b34965 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:27.714477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:27.715482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:27.715964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:27.716435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:27.728893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:27.734875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:27.735728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:27.736559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:27.738992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 125/256] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:27.740249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:27.768002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4d413-f5bd-4fed-b149-96d548081b81 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:27.771599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:27.838846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:27.839493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:27.839831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:27.840169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] PATH_INFO: `/v3/users/3d2c3e1c822840b3beac54677801f94a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:27.842442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] RBAC: Authorizing `identity:get_user(user_id=3d2c3e1c822840b3beac54677801f94a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:27.843901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:27.850209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 132/257] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/users/3d2c3e1c822840b3beac54677801f94a => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:27.851047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:27.878009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:27.921850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61d07-3caa-4269-8c32-5c873d8c0cf6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:27.970422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:27.970965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:27.971247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:27.971524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:27.972256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:27.972563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:27.973990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:27.984732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 126/258] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:27.985546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:28.109852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abeec1-ffd6-412b-92cc-88f52616f0a6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.150340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:28.151256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.151546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.151823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:28.187909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:28.199517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938abdb6-2867-4454-aa82-8c58511469d1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.233080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:28.234741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:28.343613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 127/259] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:28.344618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:28.345458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:28.346115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.346397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.346673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:28.347349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:28.347659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:28.349054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:28.365912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:28.367097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 133/260] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:28.367782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:28.421941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:28.531997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a57d-ed8e-46cb-8fe3-3111a10bcde6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.536043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c22d0-278f-4b2b-8449-15ef6548e8ee None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.578385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:28.579030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.579314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.579589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:28.581460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:28.583258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:28.585831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 134/261] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:28 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:28.586441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:28.601603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:28.603250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.603757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.604243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] PATH_INFO: `/v3/users/3d2c3e1c822840b3beac54677801f94a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:28.606320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] RBAC: Authorizing `identity:get_user(user_id=3d2c3e1c822840b3beac54677801f94a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:28.607801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:28.609641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 128/262] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:28 2022] GET /identity/v3/users/3d2c3e1c822840b3beac54677801f94a => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:28.613667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:28.733919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:28.736052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:28.745011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b520b0-bf9d-4648-8f37-875b4460d03c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.756614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35028b92-b418-4163-bd69-7c0d5e02e646 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.794158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:28.794675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b910d-6be1-4eeb-8a40-b166011ab088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:28.794869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b910d-6be1-4eeb-8a40-b166011ab088 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.795115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b910d-6be1-4eeb-8a40-b166011ab088 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.795385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b910d-6be1-4eeb-8a40-b166011ab088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:28.796023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.796137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b910d-6be1-4eeb-8a40-b166011ab088 None admin] RBAC: Adding query filter params (role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=3d2c3e1c822840b3beac54677801f94a) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:28.796410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b910d-6be1-4eeb-8a40-b166011ab088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:28.796747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.797159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] PATH_INFO: `/v3/users/dfdd91f3e0784e2f95f67e4eb703842e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:28.797894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b910d-6be1-4eeb-8a40-b166011ab088 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:28.798716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] RBAC: Authorizing `identity:get_user(user_id=dfdd91f3e0784e2f95f67e4eb703842e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:28.800620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:28.803463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 135/263] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:28 2022] GET /identity/v3/users/dfdd91f3e0784e2f95f67e4eb703842e => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:28.804385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:28.809649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 129/264] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:28 2022] GET /identity/v3/role_assignments?user.id=3d2c3e1c822840b3beac54677801f94a&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:28.811287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2918c7c-f5ff-438b-86eb-103c3af4a3cd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.811741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2918c7c-f5ff-438b-86eb-103c3af4a3cd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.814714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2918c7c-f5ff-438b-86eb-103c3af4a3cd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:28.816381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 130/265] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:28.819911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2918c7c-f5ff-438b-86eb-103c3af4a3cd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:28.955027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93b594-c68e-4ee1-b76b-a28ecf364426 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.964192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2918c7c-f5ff-438b-86eb-103c3af4a3cd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:28.995539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7c84e-bd2e-42ff-b877-82e01d8372fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:28.996245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c84e-bd2e-42ff-b877-82e01d8372fa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:28.996585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c84e-bd2e-42ff-b877-82e01d8372fa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:28.996864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c84e-bd2e-42ff-b877-82e01d8372fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.003219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e441c2e-31fe-4870-9551-a53c995944a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:29.004141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441c2e-31fe-4870-9551-a53c995944a2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.004600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441c2e-31fe-4870-9551-a53c995944a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.005146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e441c2e-31fe-4870-9551-a53c995944a2 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.006551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e441c2e-31fe-4870-9551-a53c995944a2 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:29.008284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e441c2e-31fe-4870-9551-a53c995944a2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:29.010394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 131/266] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:28 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:29.011790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b99db7-4bf0-4742-bfac-4615ba82d242 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.012219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b99db7-4bf0-4742-bfac-4615ba82d242 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.012626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b99db7-4bf0-4742-bfac-4615ba82d242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.033150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:29.033872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-91b99db7-4bf0-4742-bfac-4615ba82d242 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:29.035097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-91b99db7-4bf0-4742-bfac-4615ba82d242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:29.038493 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:29.090198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7c84e-bd2e-42ff-b877-82e01d8372fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:29.091970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7c84e-bd2e-42ff-b877-82e01d8372fa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:29.110653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 132/267] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:29.111997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b57c-6276-4d71-8370-1d6029f45382 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.112353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b57c-6276-4d71-8370-1d6029f45382 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.112812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b57c-6276-4d71-8370-1d6029f45382 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.114314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 133/268] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:29.119351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531b57c-6276-4d71-8370-1d6029f45382 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:29.213516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 136/269] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:29.214959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2094d-7e3b-4100-ad79-e6e149b3de14 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.215246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2094d-7e3b-4100-ad79-e6e149b3de14 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.215525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2094d-7e3b-4100-ad79-e6e149b3de14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.233738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-21b2094d-7e3b-4100-ad79-e6e149b3de14 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:29.234908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-21b2094d-7e3b-4100-ad79-e6e149b3de14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:29.241827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531b57c-6276-4d71-8370-1d6029f45382 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:29.278087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842e4f9-3275-45f9-81eb-7ba537cdf2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:29.278760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e4f9-3275-45f9-81eb-7ba537cdf2b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.279128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e4f9-3275-45f9-81eb-7ba537cdf2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.279400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e4f9-3275-45f9-81eb-7ba537cdf2b2 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/dfdd91f3e0784e2f95f67e4eb703842e/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.281285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842e4f9-3275-45f9-81eb-7ba537cdf2b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c320d38a1546d0a6784ac68419376d, user_id=dfdd91f3e0784e2f95f67e4eb703842e, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:29.282995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842e4f9-3275-45f9-81eb-7ba537cdf2b2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:29.294867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:29.304730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 137/270] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:29.306081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc930-01bc-4b5d-9444-27a2b68ff0e2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.306503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc930-01bc-4b5d-9444-27a2b68ff0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.306777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc930-01bc-4b5d-9444-27a2b68ff0e2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.308174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 138/271] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:29.308745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fc930-01bc-4b5d-9444-27a2b68ff0e2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:29.354076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:29.359988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 134/272] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:29 2022] PUT /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/dfdd91f3e0784e2f95f67e4eb703842e/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:29.361620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1eaec-2221-4694-8e6b-71786320895f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.361993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1eaec-2221-4694-8e6b-71786320895f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.362270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1eaec-2221-4694-8e6b-71786320895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.406769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-57b1eaec-2221-4694-8e6b-71786320895f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:29.407995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-57b1eaec-2221-4694-8e6b-71786320895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:29.449919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fc930-01bc-4b5d-9444-27a2b68ff0e2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:29.493105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:29.494112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.494402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.494681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.496103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:29.497847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:29.500100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 139/273] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:29 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:29.500739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:29.507588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 135/274] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:29.508466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1eaec-2221-4694-8e6b-71786320895f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:29.600810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:29.643524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1eaec-2221-4694-8e6b-71786320895f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:29.656872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30f2b7-c86e-4486-8fcc-931f70947392 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:29.679752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:29.686685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:29.687409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.687763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.688106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.688999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=3d2c3e1c822840b3beac54677801f94a) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:29.689322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:29.690855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:29.697398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 136/275] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:29 2022] GET /identity/v3/role_assignments?user.id=3d2c3e1c822840b3beac54677801f94a&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:29.698281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:29.714897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:29.717055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.717555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.719708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.749090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:29.762719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:29.763683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:29.764538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 140/276] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:29.769759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:29.868629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8c26a-59bb-471b-89af-0da1817751aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:29.908600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a99987-4987-46fb-bf9e-774ae37ebdf1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:29.911613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:29.912287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.912571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.912844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.915800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:29.916195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:29.918046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 137/277] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:29.918605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:29.937854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:29.962904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26360737-928b-490d-8074-256f768b3013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:29.963529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26360737-928b-490d-8074-256f768b3013 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:29.963808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26360737-928b-490d-8074-256f768b3013 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:29.964078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26360737-928b-490d-8074-256f768b3013 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:29.964917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26360737-928b-490d-8074-256f768b3013 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:29.965222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26360737-928b-490d-8074-256f768b3013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:29.967144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26360737-928b-490d-8074-256f768b3013 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:29.982171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 141/278] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:29.983371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26360737-928b-490d-8074-256f768b3013 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:30.007050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:30.049740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec625b-68a9-4573-a02a-c9d7b021a6c4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.085761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.086398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.086678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.087610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.090372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26360737-928b-490d-8074-256f768b3013 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.098055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:30.103424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:30.104295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:30.105070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 138/279] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:30.106421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:30.128320 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.128954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.129227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.129503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.130179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:30.131603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:30.214475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 142/280] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:29 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:30.215234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:30.216715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5227-96ff-458b-9f6e-c5450069f084 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.253469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:30.264574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:30.271032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.272001 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.272604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.273116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] PATH_INFO: `/v3/users/17c9be0c85e842a2b70ebf89e095f7c2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.275522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] RBAC: Authorizing `identity:get_user(user_id=17c9be0c85e842a2b70ebf89e095f7c2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:30.277653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:30.285177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 139/281] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:30 2022] GET /identity/v3/users/17c9be0c85e842a2b70ebf89e095f7c2 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:30.287185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:30.348872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1491ef-82d5-47ca-9ab6-4174445a9da6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.395794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.396745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.397219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.397823 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.399801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:30.400310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:30.402272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:30.413013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 143/282] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:30.419109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:30.427905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee2049-6de4-4fb4-a1d8-e532819e9e23 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.468503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.469362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.469811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.470229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.472436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:30.474445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:30.475803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 140/283] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:30 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:30.482154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:30.556333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:30.556722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:30.563188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dd146-8361-45bc-a331-01f3f348d18f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.600033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.600862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.601241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.601647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] PATH_INFO: `/v3/users/5a5eb1b4ae6548559bf4b7132bfa3ba2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.608756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cc2b7-dfa9-458c-a528-a04fc4e3ee0d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.616985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5eb1b4ae6548559bf4b7132bfa3ba2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:30.618773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:30.620635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 144/284] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:30 2022] GET /identity/v3/users/5a5eb1b4ae6548559bf4b7132bfa3ba2 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:30.626738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:30.649159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31868cbd-411a-44b2-b537-19547cce4217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.649970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31868cbd-411a-44b2-b537-19547cce4217 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.650251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31868cbd-411a-44b2-b537-19547cce4217 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.650529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31868cbd-411a-44b2-b537-19547cce4217 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/17c9be0c85e842a2b70ebf89e095f7c2/roles/7b93900f6d71493ca4954f9eba5d159b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.653119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31868cbd-411a-44b2-b537-19547cce4217 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c320d38a1546d0a6784ac68419376d, user_id=17c9be0c85e842a2b70ebf89e095f7c2, role_id=7b93900f6d71493ca4954f9eba5d159b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:30.654963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31868cbd-411a-44b2-b537-19547cce4217 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:30.729732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 141/285] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:55:30 2022] PUT /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d/users/17c9be0c85e842a2b70ebf89e095f7c2/roles/7b93900f6d71493ca4954f9eba5d159b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:30.746296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810517-6033-4c55-8786-4f530cfb7d59 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:30.794734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3454b8-1406-4712-a8ac-5edb76e88e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:30.795359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3454b8-1406-4712-a8ac-5edb76e88e00 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:30.795638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3454b8-1406-4712-a8ac-5edb76e88e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:30.795957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3454b8-1406-4712-a8ac-5edb76e88e00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:30.796692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3454b8-1406-4712-a8ac-5edb76e88e00 None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, user.id=5a5eb1b4ae6548559bf4b7132bfa3ba2, scope.system=all) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:30.796999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3454b8-1406-4712-a8ac-5edb76e88e00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:30.798575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3454b8-1406-4712-a8ac-5edb76e88e00 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:30.807623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 145/286] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Sep 4 15:55:30 2022] GET /identity/v3/role_assignments?user.id=5a5eb1b4ae6548559bf4b7132bfa3ba2&scope.system=all&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:31.450786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afa325-cb21-4619-ae40-44b49b2e35bd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:31.451780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afa325-cb21-4619-ae40-44b49b2e35bd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:31.452298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afa325-cb21-4619-ae40-44b49b2e35bd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:31.453942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 142/287] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:31.459323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d302520-95e9-4c1f-84d7-89961407d3ae None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:31.459801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d302520-95e9-4c1f-84d7-89961407d3ae None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:31.460010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d302520-95e9-4c1f-84d7-89961407d3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:31.483967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7d302520-95e9-4c1f-84d7-89961407d3ae None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:31.485150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7d302520-95e9-4c1f-84d7-89961407d3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:31.510135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:31.570509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 146/288] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:31.947344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afa325-cb21-4619-ae40-44b49b2e35bd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:32.079415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afa325-cb21-4619-ae40-44b49b2e35bd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:32.106063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:32.129456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:32.130457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:32.130935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:32.131401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:32.143458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:32.148756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:32.150242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:32.151121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:32.153850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 143/289] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:32.163043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d302520-95e9-4c1f-84d7-89961407d3ae None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:32.273005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d302520-95e9-4c1f-84d7-89961407d3ae None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:32.310476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:32.311451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:32.311925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:32.312385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:32.313252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:32.313833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:32.315385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:32.325802 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 147/290] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:32.337802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:32.455403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a9183-e5e6-4de3-8424-27fc3f2c6e3e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:32.490084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:32.490715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:32.490992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:32.491266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] PATH_INFO: `/v3/users/dfdd91f3e0784e2f95f67e4eb703842e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:32.492415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] RBAC: Authorizing `identity:get_user(user_id=dfdd91f3e0784e2f95f67e4eb703842e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:32.494459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:32.496447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 144/291] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:32 2022] GET /identity/v3/users/dfdd91f3e0784e2f95f67e4eb703842e => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:32.506926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:32.523604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:32.610246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551f06b-fd3d-450d-bb70-804a01873682 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:32.646190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57eed3-02e4-4f42-a982-4ef916939f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:32.646800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57eed3-02e4-4f42-a982-4ef916939f23 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:32.647080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57eed3-02e4-4f42-a982-4ef916939f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:32.647350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57eed3-02e4-4f42-a982-4ef916939f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:32.719917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57eed3-02e4-4f42-a982-4ef916939f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:32.721354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57eed3-02e4-4f42-a982-4ef916939f23 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:32.784397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:32.877529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 148/292] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:32.882838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:32.905135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610bf8b-5c79-412e-9dad-324ac0ef644f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:32.905437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610bf8b-5c79-412e-9dad-324ac0ef644f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:32.905723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610bf8b-5c79-412e-9dad-324ac0ef644f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:32.907071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 149/293] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:32.916230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a2da5-3afc-49e0-908e-ba44c2654296 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:32.916512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a2da5-3afc-49e0-908e-ba44c2654296 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:32.916785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a2da5-3afc-49e0-908e-ba44c2654296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:32.954589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-005a2da5-3afc-49e0-908e-ba44c2654296 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:32.955799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-005a2da5-3afc-49e0-908e-ba44c2654296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:32.959685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:33.022027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59065-658d-48a5-972f-375e72ebbcea None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:33.034760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 150/294] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:33.036065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce3fba-e39e-4de5-a547-3d63703584c1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.036361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce3fba-e39e-4de5-a547-3d63703584c1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.036639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce3fba-e39e-4de5-a547-3d63703584c1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.038062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 151/295] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:33.043898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167d9f-d94d-4a4e-b761-34ef73de814e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.044187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167d9f-d94d-4a4e-b761-34ef73de814e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.044469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167d9f-d94d-4a4e-b761-34ef73de814e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.063699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-85167d9f-d94d-4a4e-b761-34ef73de814e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:33.064902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-85167d9f-d94d-4a4e-b761-34ef73de814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:33.091478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:33.092234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.092523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.092806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.094712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:33.096166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:33.100095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 145/296] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:32 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:33.107189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:33.149946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:33.208069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 152/297] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:33.279436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdf9ad-e538-47bd-9ecb-f23bbeecd2e7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:33.303132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:33.319599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb5d7-3459-4068-ad77-9fb2fa3d7fb1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.319599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb5d7-3459-4068-ad77-9fb2fa3d7fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.319599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb5d7-3459-4068-ad77-9fb2fa3d7fb1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.319599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 153/298] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:33.332704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a1e0-ae0b-4331-9e2e-5273b95b9bce None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.332704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a1e0-ae0b-4331-9e2e-5273b95b9bce None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.332704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a1e0-ae0b-4331-9e2e-5273b95b9bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.332953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e090d2c-9abe-4cea-b84e-34debf97be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:33.332953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090d2c-9abe-4cea-b84e-34debf97be46 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.332953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090d2c-9abe-4cea-b84e-34debf97be46 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.332953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090d2c-9abe-4cea-b84e-34debf97be46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.333217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090d2c-9abe-4cea-b84e-34debf97be46 None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=dfdd91f3e0784e2f95f67e4eb703842e) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:33.333217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090d2c-9abe-4cea-b84e-34debf97be46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:33.333217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090d2c-9abe-4cea-b84e-34debf97be46 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:33.344803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 146/299] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:33 2022] GET /identity/v3/role_assignments?user.id=dfdd91f3e0784e2f95f67e4eb703842e&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:33.369100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4e897-d662-40e3-ae2c-dc3b81c0855e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.369406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4e897-d662-40e3-ae2c-dc3b81c0855e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.369694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4e897-d662-40e3-ae2c-dc3b81c0855e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.371190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 147/300] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:33.371685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-21f8a1e0-ae0b-4331-9e2e-5273b95b9bce None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:33.373031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-21f8a1e0-ae0b-4331-9e2e-5273b95b9bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:33.379435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a2cc-c05c-44b5-8b02-cb97977a154d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.379435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a2cc-c05c-44b5-8b02-cb97977a154d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.379659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a2cc-c05c-44b5-8b02-cb97977a154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.408221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0cd1a2cc-c05c-44b5-8b02-cb97977a154d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:33.409510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cd1a2cc-c05c-44b5-8b02-cb97977a154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:33.461611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 154/301] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:33.463392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a1e0-ae0b-4331-9e2e-5273b95b9bce None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:33.501592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 148/302] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:33.563159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1a2cc-c05c-44b5-8b02-cb97977a154d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:33.611918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a1e0-ae0b-4331-9e2e-5273b95b9bce None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:33.622129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:33.651145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:33.651846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.652126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.652433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.654251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:33.655932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:33.658600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 155/303] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:33.664377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:33.703482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1a2cc-c05c-44b5-8b02-cb97977a154d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:33.716768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:33.753448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:33.754120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.754397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.754667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.768123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:33.773892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:33.774902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:33.775796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 149/304] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:55:33.776448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:33.778130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8dba4-bee9-4eea-b5b3-8995fcdff2ce None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:33.815695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:33.816239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.816522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.816799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] PATH_INFO: `/v3/groups` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.817436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:33.818898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:33.878745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c34dbe-fcce-4e3a-88fc-7639534f355e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:33.882354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 156/305] 10.222.0.39 () {60 vars in 1134 bytes} [Sun Sep 4 15:55:33 2022] POST /identity/v3/groups => generated 229 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:33.883179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:33.919986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:33.924099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:33.924753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:33.925046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:33.925355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:33.938540 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:33.944142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:33.945130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 150/306] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:33.946888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:33.978735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.005187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e041a-c9d8-4bf3-bf0a-0f3fc013c0a7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.047975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.049049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.049049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.049251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.049898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:34.050222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.051560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.060594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 157/307] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:34.061412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.062108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c3962-a500-40dc-8398-fb68166cb1d7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.101739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.103057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.103364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.103721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.114650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:34.120062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:34.121124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 151/308] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:34.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.167937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.174481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031701ce-e87e-4f11-baf0-2ce139e3d906 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.216698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.218004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.218004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.218198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.218978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:34.219649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.221094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.226219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.230925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 158/309] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:34.231656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.241810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9239c4-41b5-4689-94b9-b24b69271dc4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.282520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.283511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.283958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.284237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] PATH_INFO: `/v3/users/72254a02d6134859b9beab72238c9e7d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.299400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] RBAC: Authorizing `identity:get_user(user_id=72254a02d6134859b9beab72238c9e7d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.300860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.302513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 152/310] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/users/72254a02d6134859b9beab72238c9e7d => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:34.306826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.346225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba5bb-e36a-4e1e-af2b-b2b9d6056815 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.383321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.384931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.385264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.385540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.386404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:34.387527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.388917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.397954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 159/311] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:34.398772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.421112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.421368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbaac6-c7b5-4d1f-b7e0-9c3c6a05426c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.460062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.460699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.460981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.461252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] PATH_INFO: `/v3/users/17c9be0c85e842a2b70ebf89e095f7c2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.463391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] RBAC: Authorizing `identity:get_user(user_id=17c9be0c85e842a2b70ebf89e095f7c2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.464885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.466611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 153/312] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/users/17c9be0c85e842a2b70ebf89e095f7c2 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:34.467379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.486630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.520886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197a75b-ef94-4e1d-b0ee-c5c5f31ca1fb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.559248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.560056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.560395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.560672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.561460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] RBAC: Adding query filter params (role.id=7b93900f6d71493ca4954f9eba5d159b, user.id=72254a02d6134859b9beab72238c9e7d, scope.system=all) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:34.561807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.563453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.577592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 160/313] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/role_assignments?user.id=72254a02d6134859b9beab72238c9e7d&scope.system=all&role.id=7b93900f6d71493ca4954f9eba5d159b => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:34.577592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.580896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0560d0c-a8d1-4af4-b6c7-de22da79eeff None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.623480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.624248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.624596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.624941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] PATH_INFO: `/v3/users/5a5eb1b4ae6548559bf4b7132bfa3ba2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.626562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5eb1b4ae6548559bf4b7132bfa3ba2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.628137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.636039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 154/314] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/users/5a5eb1b4ae6548559bf4b7132bfa3ba2 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:34.637020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:34.690263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.719275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b7de-0809-4a97-be9e-1d87187fb76b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.741076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.762894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437bf80-5d83-4336-b44b-aeb79a948f77 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:34.771983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.773289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.773902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.775207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.800426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:34.801054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:34.801334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:34.801635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] PATH_INFO: `/v3/system/users/5a5eb1b4ae6548559bf4b7132bfa3ba2/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:34.803062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a5eb1b4ae6548559bf4b7132bfa3ba2, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.804512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.856365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:34.857923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:34.882095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 155/315] 10.222.0.39 () {58 vars in 1334 bytes} [Sun Sep 4 15:55:34 2022] PUT /identity/v3/system/users/5a5eb1b4ae6548559bf4b7132bfa3ba2/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:34.958017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:34.985690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 161/316] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:34.991038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:35.106248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc94e6-b979-42ad-a6c4-8c94786f3d15 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:35.142452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeb73f-0c18-47c4-ae0c-e3fbb917631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:35.143324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb73f-0c18-47c4-ae0c-e3fbb917631f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:35.143776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb73f-0c18-47c4-ae0c-e3fbb917631f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:35.144198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb73f-0c18-47c4-ae0c-e3fbb917631f None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:35.145467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eeb73f-0c18-47c4-ae0c-e3fbb917631f None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:35.147078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eeb73f-0c18-47c4-ae0c-e3fbb917631f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:35.148909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 156/317] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:34 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:35.155271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:35.261433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e617-49ee-4532-bffb-3651a9e16058 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:35.297179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb2b0c-dff8-41d6-b0a6-d091269289fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:35.297830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb2b0c-dff8-41d6-b0a6-d091269289fd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:35.298112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb2b0c-dff8-41d6-b0a6-d091269289fd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:35.298385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb2b0c-dff8-41d6-b0a6-d091269289fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:35.299119 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb2b0c-dff8-41d6-b0a6-d091269289fd None admin] RBAC: Adding query filter params (role.id=7b93900f6d71493ca4954f9eba5d159b, scope.project.id=f2c320d38a1546d0a6784ac68419376d, user.id=17c9be0c85e842a2b70ebf89e095f7c2) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:35.299428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb2b0c-dff8-41d6-b0a6-d091269289fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:35.300861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb2b0c-dff8-41d6-b0a6-d091269289fd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:35.312285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 162/318] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:55:35 2022] GET /identity/v3/role_assignments?user.id=17c9be0c85e842a2b70ebf89e095f7c2&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=7b93900f6d71493ca4954f9eba5d159b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:35.895692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c382f3-7c1c-4dbd-b802-f43d7851df33 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:35.896197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c382f3-7c1c-4dbd-b802-f43d7851df33 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:35.896270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c382f3-7c1c-4dbd-b802-f43d7851df33 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:35.897964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 157/319] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:35.903220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32b8f8-0643-45d9-9730-73f72ea3b026 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:35.903623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32b8f8-0643-45d9-9730-73f72ea3b026 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:35.903796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32b8f8-0643-45d9-9730-73f72ea3b026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:35.922881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5d32b8f8-0643-45d9-9730-73f72ea3b026 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:35.924062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5d32b8f8-0643-45d9-9730-73f72ea3b026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:35.970068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:35.996741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 163/320] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:36.266768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c382f3-7c1c-4dbd-b802-f43d7851df33 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:36.287680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:36.382926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c382f3-7c1c-4dbd-b802-f43d7851df33 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:36.418048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23133d-b73e-43d0-9930-0cacf4b8380b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:36.418782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23133d-b73e-43d0-9930-0cacf4b8380b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:36.419067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23133d-b73e-43d0-9930-0cacf4b8380b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:36.419341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23133d-b73e-43d0-9930-0cacf4b8380b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:36.420435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23133d-b73e-43d0-9930-0cacf4b8380b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:36.421913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23133d-b73e-43d0-9930-0cacf4b8380b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:36.423646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 158/321] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:36.429042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32b8f8-0643-45d9-9730-73f72ea3b026 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:36.518314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc364999-d42f-46b7-8484-974838d3743b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:36.518539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc364999-d42f-46b7-8484-974838d3743b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:36.518814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc364999-d42f-46b7-8484-974838d3743b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:36.520174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 159/322] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:36.525392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b297e2-55b1-494f-a7b6-e95fddaf77c2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:36.525705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b297e2-55b1-494f-a7b6-e95fddaf77c2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:36.525993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b297e2-55b1-494f-a7b6-e95fddaf77c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:36.529533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32b8f8-0643-45d9-9730-73f72ea3b026 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:36.544605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-97b297e2-55b1-494f-a7b6-e95fddaf77c2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:36.545841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-97b297e2-55b1-494f-a7b6-e95fddaf77c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:36.565200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1d3d3-f790-440f-9e93-91dd0d163342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:36.565842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1d3d3-f790-440f-9e93-91dd0d163342 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:36.566127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1d3d3-f790-440f-9e93-91dd0d163342 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:36.566401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1d3d3-f790-440f-9e93-91dd0d163342 None admin] PATH_INFO: `/v3/groups` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:36.567121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1d3d3-f790-440f-9e93-91dd0d163342 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:36.568483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1d3d3-f790-440f-9e93-91dd0d163342 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:36.615932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 160/323] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:36.670898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 164/324] 10.222.0.39 () {60 vars in 1134 bytes} [Sun Sep 4 15:55:36 2022] POST /identity/v3/groups => generated 226 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:36.801208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7df881-7d72-4f05-8424-e5a4afae26bc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:36.801441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7df881-7d72-4f05-8424-e5a4afae26bc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:36.801719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7df881-7d72-4f05-8424-e5a4afae26bc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:36.803080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 161/325] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:36.808443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57da5-c719-427e-84c6-1075d9e2c302 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:36.808848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57da5-c719-427e-84c6-1075d9e2c302 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:36.809128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57da5-c719-427e-84c6-1075d9e2c302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:36.828379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cce57da5-c719-427e-84c6-1075d9e2c302 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:36.829591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cce57da5-c719-427e-84c6-1075d9e2c302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:36.862232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:36.903536 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7df881-7d72-4f05-8424-e5a4afae26bc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:36.906191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 165/326] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:36.923326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:37.022256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7df881-7d72-4f05-8424-e5a4afae26bc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:37.062893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:37.063461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:37.063743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:37.064081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:37.076171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:37.081352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:37.082357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:37.083181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 162/327] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:55:37.088770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57da5-c719-427e-84c6-1075d9e2c302 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:37.196267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:37.226012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57da5-c719-427e-84c6-1075d9e2c302 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:37.271135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:37.271913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:37.272199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:37.272471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:37.273262 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:37.273593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:37.275732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:37.289861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 166/328] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:37.295623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:37.306638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:37.342197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:37.357937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdabaae-21fc-4dfc-bcc0-7ac4c2fa471c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:37.402048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:37.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:37.402975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:37.403249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:37.417527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:37.423200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:37.424072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:37.424904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:37.426617 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 163/329] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:37.434820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:37.439406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ba317-2c9b-4667-af0a-229e8a9e780c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:37.504795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:37.505456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:37.505772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:37.506046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] PATH_INFO: `/v3/users/72254a02d6134859b9beab72238c9e7d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:37.507362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] RBAC: Authorizing `identity:get_user(user_id=72254a02d6134859b9beab72238c9e7d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:37.509399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:37.516913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 167/330] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:37 2022] GET /identity/v3/users/72254a02d6134859b9beab72238c9e7d => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:37.525120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:37.581965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e836e-b224-49d7-a2f3-b4d5b1f037cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:37.622107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:37.630431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:37.631129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:37.631414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:37.631690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:37.632372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:37.643698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:37.643698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:37.643870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:37.652785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9f9f2-ac3e-4a09-ad2f-965f5322e547 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:37.656188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 164/331] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:37.663230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:37.730085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:37.730775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:37.731061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:37.731343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] PATH_INFO: `/v3/system/users/72254a02d6134859b9beab72238c9e7d/roles/7b93900f6d71493ca4954f9eba5d159b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:37.735209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=72254a02d6134859b9beab72238c9e7d, role_id=7b93900f6d71493ca4954f9eba5d159b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:37.736610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:37.816524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65026497-9162-4443-8eed-2f72cd20f572 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:37.838552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 168/332] 10.222.0.39 () {58 vars in 1334 bytes} [Sun Sep 4 15:55:37 2022] PUT /identity/v3/system/users/72254a02d6134859b9beab72238c9e7d/roles/7b93900f6d71493ca4954f9eba5d159b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:37.862510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13152f22-42ad-4978-aabc-afe97ca85a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:37.863236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13152f22-42ad-4978-aabc-afe97ca85a27 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:37.863519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13152f22-42ad-4978-aabc-afe97ca85a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:37.863848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13152f22-42ad-4978-aabc-afe97ca85a27 None admin] PATH_INFO: `/v3/users/5a5eb1b4ae6548559bf4b7132bfa3ba2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:37.870453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13152f22-42ad-4978-aabc-afe97ca85a27 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5eb1b4ae6548559bf4b7132bfa3ba2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:37.871989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13152f22-42ad-4978-aabc-afe97ca85a27 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:37.873824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 165/333] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:37 2022] GET /identity/v3/users/5a5eb1b4ae6548559bf4b7132bfa3ba2 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:37.880388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:38.015724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6947d-2f43-4843-a58c-a02a4c7800cb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:38.053171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e717083-6f4d-4ad7-8594-c9d40dc862d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:38.053819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e717083-6f4d-4ad7-8594-c9d40dc862d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:38.054098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e717083-6f4d-4ad7-8594-c9d40dc862d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:38.054370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e717083-6f4d-4ad7-8594-c9d40dc862d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:38.055115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e717083-6f4d-4ad7-8594-c9d40dc862d2 None admin] RBAC: Adding query filter params (role.id=2d1fb653600f44dda837472d2b0cfc75, user.id=5a5eb1b4ae6548559bf4b7132bfa3ba2, scope.system=all) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:38.055452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e717083-6f4d-4ad7-8594-c9d40dc862d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:38.057100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e717083-6f4d-4ad7-8594-c9d40dc862d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:38.074739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 169/334] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Sep 4 15:55:37 2022] GET /identity/v3/role_assignments?user.id=5a5eb1b4ae6548559bf4b7132bfa3ba2&scope.system=all&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:38.993608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e9753-01bf-46e5-8edb-108ca63706f7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:38.993608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e9753-01bf-46e5-8edb-108ca63706f7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:38.993608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e9753-01bf-46e5-8edb-108ca63706f7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:38.995449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 166/335] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:39.001388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af59cae-90ad-40b1-a90c-00eb1b336339 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:39.001943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af59cae-90ad-40b1-a90c-00eb1b336339 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:39.002414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af59cae-90ad-40b1-a90c-00eb1b336339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:39.021941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3af59cae-90ad-40b1-a90c-00eb1b336339 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:39.023903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3af59cae-90ad-40b1-a90c-00eb1b336339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:39.038862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:39.099707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 170/336] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:39.394428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e9753-01bf-46e5-8edb-108ca63706f7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:39.462801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:39.505644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e9753-01bf-46e5-8edb-108ca63706f7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:39.542359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7a610-27c5-4c9a-93cd-75091501ebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:39.543346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7a610-27c5-4c9a-93cd-75091501ebe3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:39.543825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7a610-27c5-4c9a-93cd-75091501ebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:39.544333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7a610-27c5-4c9a-93cd-75091501ebe3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:39.555189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7a610-27c5-4c9a-93cd-75091501ebe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5aa7a610-27c5-4c9a-93cd-75091501ebe3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:39.561032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:39.561947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:39.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:39.565266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 167/337] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:39.572468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af59cae-90ad-40b1-a90c-00eb1b336339 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:39.678900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af59cae-90ad-40b1-a90c-00eb1b336339 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:39.690331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cda46-cf90-412b-93b4-82f632366510 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:39.690922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cda46-cf90-412b-93b4-82f632366510 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:39.691392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cda46-cf90-412b-93b4-82f632366510 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:39.692963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 168/338] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:39.698920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd747c-a2f2-42fb-9417-938be69d18f8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:39.699198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd747c-a2f2-42fb-9417-938be69d18f8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:39.699444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd747c-a2f2-42fb-9417-938be69d18f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:39.725890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:39.726929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:39.727466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:39.727929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:39.728865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:39.729361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:39.731518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:39.742588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2ffd747c-a2f2-42fb-9417-938be69d18f8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:39.743924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ffd747c-a2f2-42fb-9417-938be69d18f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:39.744716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 171/339] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:39.751929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:39.768313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:39.860957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 169/340] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:39.917796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a6761-5b51-4c42-9de2-be8818842b28 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:39.965929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:39.966465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:39.966746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:39.967019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.058958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:40.060434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:40.076828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:40.164023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd747c-a2f2-42fb-9417-938be69d18f8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:40.172059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 172/341] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:40.177985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:40.238399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:40.294196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd747c-a2f2-42fb-9417-938be69d18f8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:40.295575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd7bc2-594d-49e3-ab2e-aa2619d45bef None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:40.306289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:40.332965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:40.334250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.334840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.335384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.335703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:40.336360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.336801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.336952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.338309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:40.339873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:40.342298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 173/342] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:40 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:40.349797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:40.365481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:40.371298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:40.372399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:40.373308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:40.377914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 170/343] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:55:40.408374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:40.500412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09455d6c-0189-4782-a9f4-be4292c04cae None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:40.543468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:40.544119 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.544401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.544678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.556981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:40.558490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:40.560171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 174/344] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:40 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:40.566357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:40.611202 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46ab4-a46f-406c-b23c-c73f046f1aa6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:40.629356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:40.657531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:40.658187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad6c30-e991-4f5e-a60d-1cacd1067503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:40.659487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad6c30-e991-4f5e-a60d-1cacd1067503 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.659949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad6c30-e991-4f5e-a60d-1cacd1067503 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.660981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad6c30-e991-4f5e-a60d-1cacd1067503 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.661863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad6c30-e991-4f5e-a60d-1cacd1067503 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:40.662353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad6c30-e991-4f5e-a60d-1cacd1067503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:40.663920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad6c30-e991-4f5e-a60d-1cacd1067503 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:40.675345 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 171/345] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:40.677003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c998b4-fe31-4f69-952a-1f0f29427c66 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.677479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c998b4-fe31-4f69-952a-1f0f29427c66 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.682704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c998b4-fe31-4f69-952a-1f0f29427c66 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.687545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 172/346] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:40.687545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c998b4-fe31-4f69-952a-1f0f29427c66 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:40.769652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304863f-b13e-43bf-9323-b473f97c5f52 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:40.810704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3751ed-ea29-4ba2-8411-c536700be6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:40.811835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3751ed-ea29-4ba2-8411-c536700be6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.812304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3751ed-ea29-4ba2-8411-c536700be6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.812758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3751ed-ea29-4ba2-8411-c536700be6f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.813981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3751ed-ea29-4ba2-8411-c536700be6f7 None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=300553fe41194f45839b5a1043768774) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:40.814524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3751ed-ea29-4ba2-8411-c536700be6f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:40.816114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3751ed-ea29-4ba2-8411-c536700be6f7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:40.820965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c998b4-fe31-4f69-952a-1f0f29427c66 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:40.823156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 175/347] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:40 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=300553fe41194f45839b5a1043768774&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:40.824624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a0a5e-c367-4f58-8b2d-51c9f94febe9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.825081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a0a5e-c367-4f58-8b2d-51c9f94febe9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.825575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a0a5e-c367-4f58-8b2d-51c9f94febe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.858829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-682a0a5e-c367-4f58-8b2d-51c9f94febe9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:40.860069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-682a0a5e-c367-4f58-8b2d-51c9f94febe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:40.867469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:40.868645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:40.868925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:40.869352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:40.957364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:40.977463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:40.981159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:41.047323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 176/348] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:41.048575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f75938-66c9-492b-b056-6601317ebcd9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.048865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f75938-66c9-492b-b056-6601317ebcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.049144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f75938-66c9-492b-b056-6601317ebcd9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.050605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 177/349] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:41.056382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1e9a-30ae-407b-a271-228ecc81e9a4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.056682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1e9a-30ae-407b-a271-228ecc81e9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.056947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1e9a-30ae-407b-a271-228ecc81e9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.098143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 173/350] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:55:41.103159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dcec1e9a-30ae-407b-a271-228ecc81e9a4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:41.104374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dcec1e9a-30ae-407b-a271-228ecc81e9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:41.107325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:41.152600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:41.214579 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:41.219265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 178/351] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:41.241827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ef97-79cb-4814-93ce-94dc49f23a10 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:41.291016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d95eb-3aac-4e79-84ac-20df59c70cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:41.291658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d95eb-3aac-4e79-84ac-20df59c70cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.291936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d95eb-3aac-4e79-84ac-20df59c70cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.292210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d95eb-3aac-4e79-84ac-20df59c70cd4 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.293367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d95eb-3aac-4e79-84ac-20df59c70cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:41.294849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d95eb-3aac-4e79-84ac-20df59c70cd4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:41.297582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 174/352] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:41 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:41.302709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec1e9a-30ae-407b-a271-228ecc81e9a4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:41.319018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafeb4c-a918-4df4-ac15-6e75bf258b48 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.319335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafeb4c-a918-4df4-ac15-6e75bf258b48 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.319625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafeb4c-a918-4df4-ac15-6e75bf258b48 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.322100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 175/353] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:41.328015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79effa2c-79f6-4af1-86ff-f97dbb0607af None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.328361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79effa2c-79f6-4af1-86ff-f97dbb0607af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.328639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79effa2c-79f6-4af1-86ff-f97dbb0607af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.349201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-79effa2c-79f6-4af1-86ff-f97dbb0607af None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:41.350552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-79effa2c-79f6-4af1-86ff-f97dbb0607af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:41.425894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 176/354] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:55:41.450459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec1e9a-30ae-407b-a271-228ecc81e9a4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:41.499432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:41.505900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.505900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.505900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.506952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:41.508349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:41.510376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 179/355] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:41 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:41.515900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79effa2c-79f6-4af1-86ff-f97dbb0607af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:41.567692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:41.623138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:41.632064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:41.678221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79effa2c-79f6-4af1-86ff-f97dbb0607af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:41.707678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402623f-af6a-474f-89a7-c43eb1c33ec9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:41.734529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:41.735206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.735488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.735758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.736487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=300553fe41194f45839b5a1043768774) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:41.736791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:41.738501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:41.751327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 177/356] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:41 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=300553fe41194f45839b5a1043768774&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:41.752178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:41.765346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:41.766095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:41.766376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:41.766647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:41.787957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:41.793077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:41.794018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:41.794807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 180/357] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:41.800241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:41.935345 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ff5ed-bcbf-4d41-ae39-a81bf2e254ab None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:41.944932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:41.959649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fb54-8d3d-4e21-962b-cc6f33f128b5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:41.999852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.000777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.001590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.002016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.005353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:42.015896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.017652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.023396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.023396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.023396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:42.023396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:42.023396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:42.031825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:42.035063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 181/358] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:42.035938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:42.038010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:42.038985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:42.040496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:42.044062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 178/359] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:42.045102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:42.169766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ddc6-f08f-438d-8c23-f4363f7a3bb0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:42.213677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.214663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.215566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.216048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] PATH_INFO: `/v3/users/72254a02d6134859b9beab72238c9e7d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.217428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] RBAC: Authorizing `identity:get_user(user_id=72254a02d6134859b9beab72238c9e7d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:42.219089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:42.221556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 179/360] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:55:42 2022] GET /identity/v3/users/72254a02d6134859b9beab72238c9e7d => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:42.222742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:42.236532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29273a23-241c-4e72-8217-1a050ce323f8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:42.286182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9f789-eaa9-41f2-9f0b-06f89596138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.287096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9f789-eaa9-41f2-9f0b-06f89596138d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.287539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9f789-eaa9-41f2-9f0b-06f89596138d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.287971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9f789-eaa9-41f2-9f0b-06f89596138d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.299710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9f789-eaa9-41f2-9f0b-06f89596138d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:42.302779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-72f9f789-eaa9-41f2-9f0b-06f89596138d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:42.307297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:42.308426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 182/361] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:42.319095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7786-0a70-4e5e-90fc-817febd916d7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.320761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7786-0a70-4e5e-90fc-817febd916d7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.320761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7786-0a70-4e5e-90fc-817febd916d7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.320761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 183/362] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:42.320761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf7786-0a70-4e5e-90fc-817febd916d7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:42.351985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:42.382628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fdc9-66c5-4849-94a9-7d0c0599081b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:42.427224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.427832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.428119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.428395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.429066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:42.429374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:42.431587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:42.440728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 180/363] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:42.441671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:42.462558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf7786-0a70-4e5e-90fc-817febd916d7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:42.514359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3537b8-adca-4796-881c-db8f4da663f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.514909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3537b8-adca-4796-881c-db8f4da663f4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.515187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3537b8-adca-4796-881c-db8f4da663f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.515458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3537b8-adca-4796-881c-db8f4da663f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.516192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3537b8-adca-4796-881c-db8f4da663f4 None admin] RBAC: Adding query filter params (role.id=7b93900f6d71493ca4954f9eba5d159b, user.id=72254a02d6134859b9beab72238c9e7d, scope.system=all) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:42.516559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3537b8-adca-4796-881c-db8f4da663f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:42.518328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3537b8-adca-4796-881c-db8f4da663f4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:42.533796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 184/364] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Sep 4 15:55:42 2022] GET /identity/v3/role_assignments?user.id=72254a02d6134859b9beab72238c9e7d&scope.system=all&role.id=7b93900f6d71493ca4954f9eba5d159b => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:42.535131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402bdd2-9456-41f3-a71b-093f1e004df3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.535423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402bdd2-9456-41f3-a71b-093f1e004df3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.535704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402bdd2-9456-41f3-a71b-093f1e004df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.563953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5402bdd2-9456-41f3-a71b-093f1e004df3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:42.565224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5402bdd2-9456-41f3-a71b-093f1e004df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:42.626278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbad4bc-88bb-49fa-8c7f-e2ba199af544 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:42.661061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 185/365] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:42.662024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402bdd2-9456-41f3-a71b-093f1e004df3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:42.667400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:42.673413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:42.681494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beede758-521c-4448-8727-654e63d59160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.682123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beede758-521c-4448-8727-654e63d59160 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.682399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beede758-521c-4448-8727-654e63d59160 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.686930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beede758-521c-4448-8727-654e63d59160 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.686930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beede758-521c-4448-8727-654e63d59160 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:42.686930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beede758-521c-4448-8727-654e63d59160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:42.686930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beede758-521c-4448-8727-654e63d59160 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:42.700791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 181/366] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:42.708159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beede758-521c-4448-8727-654e63d59160 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:42.782793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402bdd2-9456-41f3-a71b-093f1e004df3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:42.821267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.822285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.822461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.822704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.823164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beede758-521c-4448-8727-654e63d59160 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:42.864808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:42.865485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:42.865800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:42.866074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:42.900240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:42.901850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:42.931506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:42.946630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:42.948094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:42.953786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:43.015629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 186/367] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:43.027648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.057960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 182/368] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:55:43.063816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.196577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:43.201722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1745681-e3e9-48b1-abeb-55e10d424022 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.219755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c62c27-9088-42e6-9a52-0113dde2e95d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.224172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:43.267795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.269234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.269812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.270263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.271674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:43.273253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:43.275561 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 187/369] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:43.276285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.279571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.280232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.280517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.280793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.286279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:43.287747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:43.289778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 183/370] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:43.291157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.399238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfdbf-7037-46d3-88bc-1096d9b9d4ff None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.438214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b5eab-da97-4cb0-917a-957bfe8b0bc4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.448680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.449328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.449659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.449923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.462283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:43.469341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:43.470266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:43.471060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 188/371] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:43.471721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.489154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.490278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.490760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.491323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.492873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:43.495266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:43.497532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 184/372] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:43.498569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.517328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:43.535036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:43.616885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835f2a7-3147-49cd-af6f-287cabc76267 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.638879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa111c5-7e0e-44fe-abb6-878d0ef926a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.661032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.662131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.662420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.662694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.663917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:43.665680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:43.667268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 189/373] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:43.668121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.682519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c298-f10f-4896-9305-402276f9ef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.683637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c298-f10f-4896-9305-402276f9ef0d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.683932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c298-f10f-4896-9305-402276f9ef0d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.684213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c298-f10f-4896-9305-402276f9ef0d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.685179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c298-f10f-4896-9305-402276f9ef0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:43.685488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c298-f10f-4896-9305-402276f9ef0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:43.687628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c298-f10f-4896-9305-402276f9ef0d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:43.699026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 185/374] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:43.701380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150ff3-3ea3-4ec0-bc93-1d7727437e6e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.701777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150ff3-3ea3-4ec0-bc93-1d7727437e6e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.702250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150ff3-3ea3-4ec0-bc93-1d7727437e6e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.703615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 186/375] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:43.707670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87150ff3-3ea3-4ec0-bc93-1d7727437e6e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.807668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59a74c-fd54-49ed-a284-e637fae4a6a9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.810049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:43.830819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:43.849793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.850180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87150ff3-3ea3-4ec0-bc93-1d7727437e6e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:43.851929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.851929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.851929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.855085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=f2c320d38a1546d0a6784ac68419376d) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:43.855085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:43.855956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:43.864094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 190/376] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:43.864916 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:43.911950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e2f87-986b-475e-8dae-fcbeac95578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:43.912916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e2f87-986b-475e-8dae-fcbeac95578f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.913419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e2f87-986b-475e-8dae-fcbeac95578f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.913956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e2f87-986b-475e-8dae-fcbeac95578f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.915018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e2f87-986b-475e-8dae-fcbeac95578f None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=f2c320d38a1546d0a6784ac68419376d) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:43.915703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e2f87-986b-475e-8dae-fcbeac95578f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:43.917644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e2f87-986b-475e-8dae-fcbeac95578f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:43.925149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 187/377] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:43.926462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a54adb-6253-4ccb-a775-8e2e070aa627 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:43.926751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a54adb-6253-4ccb-a775-8e2e070aa627 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:43.927029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a54adb-6253-4ccb-a775-8e2e070aa627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:43.971694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-74a54adb-6253-4ccb-a775-8e2e070aa627 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:43.974623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74a54adb-6253-4ccb-a775-8e2e070aa627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:44.087934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 188/378] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:44.088622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de644fbc-d727-4e06-9ef3-4f07cd36d2be None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:44.129609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:44.130239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:44.130520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:44.130797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:44.165927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:44.208046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:44.208046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:44.355747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 191/379] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:44.361634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a54adb-6253-4ccb-a775-8e2e070aa627 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:44.486155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:44.507443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a54adb-6253-4ccb-a775-8e2e070aa627 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:44.512350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:44.549511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651dc5e-a5b2-445e-8974-5838a9ac30d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:44.550185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651dc5e-a5b2-445e-8974-5838a9ac30d7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:44.550469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651dc5e-a5b2-445e-8974-5838a9ac30d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:44.550747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651dc5e-a5b2-445e-8974-5838a9ac30d7 None admin] PATH_INFO: `/v3/projects/677669b803a2416fad6430932644a341` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:44.551885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651dc5e-a5b2-445e-8974-5838a9ac30d7 None admin] RBAC: Authorizing `identity:get_project(project_id=677669b803a2416fad6430932644a341)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:44.553367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651dc5e-a5b2-445e-8974-5838a9ac30d7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:44.556612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 189/380] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:44 2022] GET /identity/v3/projects/677669b803a2416fad6430932644a341 => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:44.557965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42830-7493-4a86-9796-64120135b18b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:44.558255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42830-7493-4a86-9796-64120135b18b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:44.558533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce42830-7493-4a86-9796-64120135b18b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:44.559946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 190/381] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:44.562386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42830-7493-4a86-9796-64120135b18b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:44.601460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:44.649511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd17753-05ae-41a1-a64b-d75e4bc77129 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:44.691172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e742307-b029-46ce-bf1c-abdabddd14e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:44.691821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742307-b029-46ce-bf1c-abdabddd14e2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:44.692101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742307-b029-46ce-bf1c-abdabddd14e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:44.692375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742307-b029-46ce-bf1c-abdabddd14e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:44.707725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e742307-b029-46ce-bf1c-abdabddd14e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:44.710881 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce42830-7493-4a86-9796-64120135b18b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1e742307-b029-46ce-bf1c-abdabddd14e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:44.713640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:44.714537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 192/382] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:44.715337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6e11f-5140-4134-8262-727904cd21dd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:44.716074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6e11f-5140-4134-8262-727904cd21dd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:44.716074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6e11f-5140-4134-8262-727904cd21dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:44.737252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b9c6e11f-5140-4134-8262-727904cd21dd None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:44.738500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b9c6e11f-5140-4134-8262-727904cd21dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:44.759681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:44.760349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:44.760599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:44.760872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] PATH_INFO: `/v3/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:44.769855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] RBAC: Authorizing `identity:get_group(group_id=cf9f0b3b4b08421f997c03d9a6cb6bbc)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:44.771425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:44.773431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 191/383] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:44 2022] GET /identity/v3/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:44.774284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:44.796650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:44.830064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 193/384] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:44.831001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6e11f-5140-4134-8262-727904cd21dd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:44.931077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b24d-a1f6-4ead-8b6b-fe65a7923362 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:44.936984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:45.013956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.014462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.014745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.015144 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.016240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:45.016786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:45.018316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:45.036683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6e11f-5140-4134-8262-727904cd21dd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:45.041104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 192/385] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:45.046612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:45.126312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.126312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.126312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.126312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.126725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] RBAC: Adding query filter params (group.id=cf9f0b3b4b08421f997c03d9a6cb6bbc, role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=677669b803a2416fad6430932644a341) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:45.126725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:45.126725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:45.135691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 194/386] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:44 2022] GET /identity/v3/role_assignments?group.id=cf9f0b3b4b08421f997c03d9a6cb6bbc&scope.project.id=677669b803a2416fad6430932644a341&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:45.149938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:45.187416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26570566-3417-42cd-8c47-b390795d6359 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:45.233514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:45.236041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.236968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.237489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.238007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.239574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:45.241307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:45.245164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 193/387] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:45 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:45.251839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:45.412305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b238b-c53a-49bf-87f7-708bea4eb3f4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:45.424190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01a609-3226-415d-85f7-54f4f0897d18 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:45.433118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:45.467972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.469487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.469487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.469487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.491040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:45.502384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.503043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.503325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.503601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.504848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:45.506332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:45.508848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 194/388] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:45 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:45.509159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:45.510079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:45.510852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 195/389] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:45.514517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:45.524647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:45.550622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:45.672988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3da9-297b-4cb3-8a68-2b431b8d24fe None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:45.706594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6058c3b-cdc5-4291-bdd5-b9310fca20ee None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:45.714989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.715678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.715968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.716249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774/groups/b876cec3dc56414cbbce157e60155a70/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.722946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] RBAC: Authorizing `identity:create_grant(project_id=300553fe41194f45839b5a1043768774, group_id=b876cec3dc56414cbbce157e60155a70, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:45.724485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:45.756459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.757470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.758025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.759041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.760164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:45.760665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:45.762260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:45.775784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 196/390] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:45.781500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:45.826555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:45.847163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 195/391] 10.222.0.39 () {58 vars in 1442 bytes} [Sun Sep 4 15:55:45 2022] PUT /identity/v3/projects/300553fe41194f45839b5a1043768774/groups/b876cec3dc56414cbbce157e60155a70/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:45.898347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0c64-df9a-4ec8-bb85-87119787daa9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:45.945161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:45.945857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:45.946143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:45.946407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:45.947655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:45.949090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:45.950876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 197/392] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:45 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:45.956225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:46.040338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:46.075523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d4be8-9671-451b-95b4-df170a10a35c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:46.119916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6450f-b28a-4493-847a-ee15145ea6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:46.120563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6450f-b28a-4493-847a-ee15145ea6eb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:46.120845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6450f-b28a-4493-847a-ee15145ea6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:46.121123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6450f-b28a-4493-847a-ee15145ea6eb None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:46.122916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6450f-b28a-4493-847a-ee15145ea6eb None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:46.124395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6450f-b28a-4493-847a-ee15145ea6eb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:46.126642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 196/393] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:45 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:46.132384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:46.250785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89504d5f-6b56-4a9d-b4e0-ec32c85b30b2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:46.263572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:46.298560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b82bd9-8585-4ac5-9fa0-b666cea8a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:46.299237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82bd9-8585-4ac5-9fa0-b666cea8a858 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:46.299513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82bd9-8585-4ac5-9fa0-b666cea8a858 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:46.299784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82bd9-8585-4ac5-9fa0-b666cea8a858 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774/groups/b876cec3dc56414cbbce157e60155a70/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:46.301952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b82bd9-8585-4ac5-9fa0-b666cea8a858 None admin] RBAC: Authorizing `identity:create_grant(project_id=300553fe41194f45839b5a1043768774, group_id=b876cec3dc56414cbbce157e60155a70, role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:46.303378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b82bd9-8585-4ac5-9fa0-b666cea8a858 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:46.378367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 198/394] 10.222.0.39 () {58 vars in 1442 bytes} [Sun Sep 4 15:55:46 2022] PUT /identity/v3/projects/300553fe41194f45839b5a1043768774/groups/b876cec3dc56414cbbce157e60155a70/roles/eb0cba906bfe429883619da6e985a2b4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:46.386099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bef3a-2743-40e9-8100-4b6ed1786a68 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:46.386402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bef3a-2743-40e9-8100-4b6ed1786a68 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:46.386657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bef3a-2743-40e9-8100-4b6ed1786a68 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:46.388067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 197/395] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:46.393866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aca1c-75fb-4926-ae27-ad459ba9a86a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:46.394173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aca1c-75fb-4926-ae27-ad459ba9a86a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:46.394420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aca1c-75fb-4926-ae27-ad459ba9a86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:46.415113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-952aca1c-75fb-4926-ae27-ad459ba9a86a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:46.416358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-952aca1c-75fb-4926-ae27-ad459ba9a86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:46.491547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 199/396] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:46.734896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23f9a-c2e8-43e9-9b03-11970560ef17 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:46.735402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23f9a-c2e8-43e9-9b03-11970560ef17 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:46.735476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23f9a-c2e8-43e9-9b03-11970560ef17 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:46.737163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 198/397] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:46.742220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91ea3-b539-46c1-aa09-f2f03e02d961 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:46.744255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91ea3-b539-46c1-aa09-f2f03e02d961 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:46.744255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91ea3-b539-46c1-aa09-f2f03e02d961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:46.767032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-eea91ea3-b539-46c1-aa09-f2f03e02d961 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:46.768526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eea91ea3-b539-46c1-aa09-f2f03e02d961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:46.800053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23f9a-c2e8-43e9-9b03-11970560ef17 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:46.863169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 200/398] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:46.923104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23f9a-c2e8-43e9-9b03-11970560ef17 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:46.963250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:46.963250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:46.963250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:46.963250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:46.968733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:46.974193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:46.980285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:46.985289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:46.986194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 199/399] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:55:46.988463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91ea3-b539-46c1-aa09-f2f03e02d961 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.022114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:47.100843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91ea3-b539-46c1-aa09-f2f03e02d961 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:47.139204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:47.140140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.140601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.141053 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.141941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:47.142494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:47.144056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:47.153582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 201/400] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:47.159371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.268046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2df265-1876-4704-918c-b5e6cabc2799 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:47.306236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:47.307211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.307673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.308127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.310063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.312975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:47.315096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:47.316452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 200/401] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:47 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:47.325000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.431892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:47.443251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af6a5-0a41-473d-af34-b9254da44fa7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:47.484125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:47.484953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.485233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.485504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.497528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:47.501427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:47.508730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:47.514302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:47.515155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 202/402] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:47.517274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.571918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517ba0-7186-4457-9810-003f308362c8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:47.633295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e75de-13b2-42a3-aa3b-a3f0f0b7d474 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:47.640160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83d10-10e8-466e-b3b1-e1436c0d57a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:47.641076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83d10-10e8-466e-b3b1-e1436c0d57a8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.641936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83d10-10e8-466e-b3b1-e1436c0d57a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.642214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83d10-10e8-466e-b3b1-e1436c0d57a8 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.646233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83d10-10e8-466e-b3b1-e1436c0d57a8 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:47.646375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83d10-10e8-466e-b3b1-e1436c0d57a8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:47.648151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 201/403] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:47 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:47.649893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d3bdb-e849-4ac8-90df-0bf0dffd9510 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.650173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d3bdb-e849-4ac8-90df-0bf0dffd9510 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.650600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d3bdb-e849-4ac8-90df-0bf0dffd9510 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.651917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 202/404] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:47.657133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d480-3e6c-4402-a633-1f01149e5d4e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.657417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d480-3e6c-4402-a633-1f01149e5d4e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.657717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d480-3e6c-4402-a633-1f01149e5d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.676967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f540d480-3e6c-4402-a633-1f01149e5d4e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:47.678181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f540d480-3e6c-4402-a633-1f01149e5d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:47.688602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:47.689535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.690085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.690569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.691433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:47.691940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:47.693585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:47.702999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 203/405] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:47.711918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.727907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:47.757222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 203/406] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:47.757926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d480-3e6c-4402-a633-1f01149e5d4e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.818968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566086a1-1944-4fb9-b9e3-8e42faf8df9c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:47.828017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:47.855825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:47.856888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.857029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.857373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d/groups/b876cec3dc56414cbbce157e60155a70/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.859447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c320d38a1546d0a6784ac68419376d, group_id=b876cec3dc56414cbbce157e60155a70, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:47.861091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:47.870918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d480-3e6c-4402-a633-1f01149e5d4e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:47.907496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:47.908384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:47.908815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:47.909239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:47.910593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:47.912499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:47.921685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 204/407] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:47 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:47.922831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:47.947054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 204/408] 10.222.0.39 () {58 vars in 1442 bytes} [Sun Sep 4 15:55:47 2022] PUT /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d/groups/b876cec3dc56414cbbce157e60155a70/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:48.058805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92d8f1-f5de-4c09-9e8c-a34290e0a9f1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:48.095303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:48.101116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:48.101854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.102147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.102487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.103989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:48.105595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:48.110854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 205/409] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:47 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:48.113982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:48.127488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:48.220444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:48.262560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23e77-32ae-4c4c-85b7-e8711e4fd783 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:48.267123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b96a-c258-469a-9000-fe6429e46542 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:48.329447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:48.330448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.330894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.331325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.341077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:48.341929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.342278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.342608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d/groups/b876cec3dc56414cbbce157e60155a70/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.354385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c320d38a1546d0a6784ac68419376d, group_id=b876cec3dc56414cbbce157e60155a70, role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:48.356092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:48.356351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:48.363947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:48.364823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:48.365644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:48.369024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 206/410] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:48.377584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:48.441749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:48.480303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 205/411] 10.222.0.39 () {58 vars in 1442 bytes} [Sun Sep 4 15:55:48 2022] PUT /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d/groups/b876cec3dc56414cbbce157e60155a70/roles/eb0cba906bfe429883619da6e985a2b4 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:48.497534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613755a-ab2e-4b9a-9baa-4d692a689568 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:48.563895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608d477-015a-44ae-bb21-615a43b30f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:48.564801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608d477-015a-44ae-bb21-615a43b30f42 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.565263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608d477-015a-44ae-bb21-615a43b30f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.565721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608d477-015a-44ae-bb21-615a43b30f42 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.566595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608d477-015a-44ae-bb21-615a43b30f42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:48.567080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608d477-015a-44ae-bb21-615a43b30f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:48.568671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608d477-015a-44ae-bb21-615a43b30f42 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:48.582104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 207/412] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:48.588238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:48.721335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe1bf6-4725-4a5a-9d9c-2cd79eab4ff3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:48.752374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfefb8-9373-41b9-8dac-e99a81f9b7eb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.752652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfefb8-9373-41b9-8dac-e99a81f9b7eb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.752936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfefb8-9373-41b9-8dac-e99a81f9b7eb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.755351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 208/413] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:48.761126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b33b8-e593-4caf-9da4-09c6ac8fd042 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.761753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b33b8-e593-4caf-9da4-09c6ac8fd042 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.761753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b33b8-e593-4caf-9da4-09c6ac8fd042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.762149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:48.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:48.771632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.771632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.771632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] PATH_INFO: `/v3/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.771632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] RBAC: Authorizing `identity:get_group(group_id=cf9f0b3b4b08421f997c03d9a6cb6bbc)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:48.772128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:48.774113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 206/414] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:48 2022] GET /identity/v3/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:48.780100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:48.796991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9e8b33b8-e593-4caf-9da4-09c6ac8fd042 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:48.801858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e8b33b8-e593-4caf-9da4-09c6ac8fd042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:48.906852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced3ea2-0e54-405d-97ee-f974c793de8d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:48.948976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 209/415] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:48.949856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc22ec-a311-45b4-9341-6fcc12a01301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:48.950806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc22ec-a311-45b4-9341-6fcc12a01301 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:48.951263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc22ec-a311-45b4-9341-6fcc12a01301 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:48.951698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc22ec-a311-45b4-9341-6fcc12a01301 None admin] PATH_INFO: `/v3/projects/677669b803a2416fad6430932644a341` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:48.953052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc22ec-a311-45b4-9341-6fcc12a01301 None admin] RBAC: Authorizing `identity:get_project(project_id=677669b803a2416fad6430932644a341)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:48.954708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc22ec-a311-45b4-9341-6fcc12a01301 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:48.961889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 207/416] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:48 2022] GET /identity/v3/projects/677669b803a2416fad6430932644a341 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:48.963661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b33b8-e593-4caf-9da4-09c6ac8fd042 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:49.024650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:49.096933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b33b8-e593-4caf-9da4-09c6ac8fd042 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:49.126826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0044fd5-3a0a-4420-871e-49e4b44540a7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:49.127416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0044fd5-3a0a-4420-871e-49e4b44540a7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:49.127911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0044fd5-3a0a-4420-871e-49e4b44540a7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:49.134342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 208/417] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:55:49.135902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671a65a-2ce3-4b9a-a52f-5d9419de763c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:49.136393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671a65a-2ce3-4b9a-a52f-5d9419de763c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:49.136942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671a65a-2ce3-4b9a-a52f-5d9419de763c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:49.137271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:49.138018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:49.138304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:49.138585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] PATH_INFO: `/v3/projects/677669b803a2416fad6430932644a341/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:49.140538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] RBAC: Authorizing `identity:create_grant(project_id=677669b803a2416fad6430932644a341, group_id=cf9f0b3b4b08421f997c03d9a6cb6bbc, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:49.142192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:49.161636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a671a65a-2ce3-4b9a-a52f-5d9419de763c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:49.163240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a671a65a-2ce3-4b9a-a52f-5d9419de763c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:49.230149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:49.234304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 209/418] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:49.252066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 210/419] 10.222.0.39 () {58 vars in 1442 bytes} [Sun Sep 4 15:55:48 2022] PUT /identity/v3/projects/677669b803a2416fad6430932644a341/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:55:49.296741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671a65a-2ce3-4b9a-a52f-5d9419de763c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:49.450905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671a65a-2ce3-4b9a-a52f-5d9419de763c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:49.494835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:49.495472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:49.495767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:49.496049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:49.507314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:49.513330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:49.515379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:49.516196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 210/420] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:49.526702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:49.621621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:49.686939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:49.697092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7c8c9-dbad-44e7-bda2-39d51de24b34 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:49.725067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:49.760204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:49.760830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:49.761114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:49.761413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:49.762122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:49.762430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:49.764236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:49.767186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb2bbd-e6d7-48f5-bdb7-0ae60c323772 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:49.774263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 211/421] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:49.780554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:49.825490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:49.826172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:49.826455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:49.826729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:49.854314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:49.861229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:49.862219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:49.862997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:49.865124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 211/422] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:55:49.871496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:49.962905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942a8e0-77a6-43fa-ad95-8807c8482346 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:49.999970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:50.000642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.001184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.001469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.056098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aec46-4242-41a0-8cc8-c2990f005629 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:50.087518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:50.089763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:50.119119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:50.120716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:50.126533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:50.127168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.127451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.127728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.128408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:50.128718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:50.130140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:50.149703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 212/423] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:50.155465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:50.314552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 212/424] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:50.325424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:50.354733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f4320-f526-4f8f-930e-95dd56f0bb3f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:50.404770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31cac9-9aaf-4b40-a11c-f8d71b017c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:50.405268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31cac9-9aaf-4b40-a11c-f8d71b017c8e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.405547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31cac9-9aaf-4b40-a11c-f8d71b017c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.405878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31cac9-9aaf-4b40-a11c-f8d71b017c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.430717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:50.456859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e00da-72a4-4744-8e07-3f362b1b78ab None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:50.476539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:50.495201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71591d2a-afee-4a2c-9155-430992584626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:50.496209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71591d2a-afee-4a2c-9155-430992584626 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.496690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71591d2a-afee-4a2c-9155-430992584626 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.497160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71591d2a-afee-4a2c-9155-430992584626 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.498566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71591d2a-afee-4a2c-9155-430992584626 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:50.500215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71591d2a-afee-4a2c-9155-430992584626 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:50.500661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31cac9-9aaf-4b40-a11c-f8d71b017c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:50.502388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31cac9-9aaf-4b40-a11c-f8d71b017c8e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:50.503483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 213/425] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:50 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:50.504947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c7b2-6be6-45f1-b004-fe35c75b304d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.505432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c7b2-6be6-45f1-b004-fe35c75b304d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.505959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c7b2-6be6-45f1-b004-fe35c75b304d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.507711 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 214/426] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:50.508673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937c7b2-6be6-45f1-b004-fe35c75b304d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:50.620758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 213/427] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:55:50.627329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c951-e4a5-4de2-ae2c-9d7546f9cf92 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.627861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c951-e4a5-4de2-ae2c-9d7546f9cf92 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.628347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c951-e4a5-4de2-ae2c-9d7546f9cf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.643829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937c7b2-6be6-45f1-b004-fe35c75b304d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:50.647820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3863c951-e4a5-4de2-ae2c-9d7546f9cf92 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:50.649774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3863c951-e4a5-4de2-ae2c-9d7546f9cf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:50.687690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:50.688674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.689368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.689889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.691237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:50.692899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:50.694684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 215/428] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:50 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:50.695759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:50.710938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:50.729647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 214/429] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:50.730778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c951-e4a5-4de2-ae2c-9d7546f9cf92 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:50.750661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:50.858849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417bd71-4927-4a36-bd56-b66798ef6df7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:50.909436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4349222-e17b-4e3d-b0a1-dfcb78c60eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:50.910033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4349222-e17b-4e3d-b0a1-dfcb78c60eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.910308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4349222-e17b-4e3d-b0a1-dfcb78c60eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.910577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4349222-e17b-4e3d-b0a1-dfcb78c60eb8 None admin] PATH_INFO: `/v3/projects/300553fe41194f45839b5a1043768774` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.915578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4349222-e17b-4e3d-b0a1-dfcb78c60eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=300553fe41194f45839b5a1043768774)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:50.917017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4349222-e17b-4e3d-b0a1-dfcb78c60eb8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:50.921918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 216/430] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:50 2022] GET /identity/v3/projects/300553fe41194f45839b5a1043768774 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:50.927619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954b007-6416-4d6b-9edd-9cefa9aa1699 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.927903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954b007-6416-4d6b-9edd-9cefa9aa1699 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.928173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954b007-6416-4d6b-9edd-9cefa9aa1699 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.929603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 217/431] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:50.930805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954b007-6416-4d6b-9edd-9cefa9aa1699 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:50.931875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c951-e4a5-4de2-ae2c-9d7546f9cf92 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:50.969933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6300cc-22fa-4828-b5f6-c31b9766f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:50.970423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6300cc-22fa-4828-b5f6-c31b9766f34a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.970716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6300cc-22fa-4828-b5f6-c31b9766f34a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.971014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6300cc-22fa-4828-b5f6-c31b9766f34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:50.971684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6300cc-22fa-4828-b5f6-c31b9766f34a None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=300553fe41194f45839b5a1043768774) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:50.971986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6300cc-22fa-4828-b5f6-c31b9766f34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:50.973518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6300cc-22fa-4828-b5f6-c31b9766f34a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:50.980177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 215/432] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:50 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=300553fe41194f45839b5a1043768774&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 611 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:50.981458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecef02d-14b5-4811-9d6d-d85b32adecc8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:50.981828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecef02d-14b5-4811-9d6d-d85b32adecc8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:50.982101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecef02d-14b5-4811-9d6d-d85b32adecc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.003568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aecef02d-14b5-4811-9d6d-d85b32adecc8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:51.004492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aecef02d-14b5-4811-9d6d-d85b32adecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:51.035563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.075914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954b007-6416-4d6b-9edd-9cefa9aa1699 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.091987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 216/433] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:51.093102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecef02d-14b5-4811-9d6d-d85b32adecc8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.101981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.116022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.116732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.117016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.121820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.121820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:51.121820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:51.122576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 218/434] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:50 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:51.129051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.215209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecef02d-14b5-4811-9d6d-d85b32adecc8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.236342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd63b9-a4f6-4543-88da-cc8bd3447b6d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.255504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.256152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.256437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.256715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.268228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:51.274182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:51.275220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 15:55:51.276001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:51.276590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.276590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.276590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.276796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 217/435] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:51.276867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.276867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=300553fe41194f45839b5a1043768774) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:51.276997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:51.279442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:51.280951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.293752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 219/436] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=300553fe41194f45839b5a1043768774&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:51.366003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.404608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02dfc8-cad4-4342-9ca8-03d2479d9d83 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.429426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.446728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.448801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.449455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.449795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.450056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.450746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:51.451054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:51.452458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:51.463068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 218/437] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:51.469764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.580845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf434a-dd28-4d3f-b4dc-ae2ea7225b91 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.606265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aaace-9a8d-4506-a5f3-9b62ac2ab226 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.627012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542407a5-05db-49a4-9d03-712cc0f1c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.627647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542407a5-05db-49a4-9d03-712cc0f1c958 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.627925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542407a5-05db-49a4-9d03-712cc0f1c958 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.628661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542407a5-05db-49a4-9d03-712cc0f1c958 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.635353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.641965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542407a5-05db-49a4-9d03-712cc0f1c958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-542407a5-05db-49a4-9d03-712cc0f1c958 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:51.647954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:51.650867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:51.651702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 220/438] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:51.652371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.652371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.652529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb70f4e-21c3-437a-b1aa-9f0145ff1ef5 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.652599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.652665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb70f4e-21c3-437a-b1aa-9f0145ff1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.652730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.652795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb70f4e-21c3-437a-b1aa-9f0145ff1ef5 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.652795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 221/439] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:51.654487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb70f4e-21c3-437a-b1aa-9f0145ff1ef5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.742035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:51.743626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:51.745161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.784210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb70f4e-21c3-437a-b1aa-9f0145ff1ef5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.824379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e9457-5f50-4035-8568-770e9002a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.824987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e9457-5f50-4035-8568-770e9002a447 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.825269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e9457-5f50-4035-8568-770e9002a447 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.825539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e9457-5f50-4035-8568-770e9002a447 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.826226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e9457-5f50-4035-8568-770e9002a447 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:51.826529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e9457-5f50-4035-8568-770e9002a447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:51.827897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e9457-5f50-4035-8568-770e9002a447 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:51.837009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 222/440] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:51.839126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6e50f-28a1-4fde-a17b-e72ce1b400c3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.839577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6e50f-28a1-4fde-a17b-e72ce1b400c3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.839858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6e50f-28a1-4fde-a17b-e72ce1b400c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:51.854942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 219/441] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:55:51.855771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.857866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-42e6e50f-28a1-4fde-a17b-e72ce1b400c3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:51.859116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-42e6e50f-28a1-4fde-a17b-e72ce1b400c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:51.882488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.924604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 223/442] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:51.925449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6e50f-28a1-4fde-a17b-e72ce1b400c3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:51.960220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dad93a-4cdc-43d9-820e-aec0d29b7d00 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:51.995430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:51.996213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:51.997030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:51.997488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:51.997848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.035281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6e50f-28a1-4fde-a17b-e72ce1b400c3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.069675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.070303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.070645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.070807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.070951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.072025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.072228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.073452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.075107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 224/443] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:52.080555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.100397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:52.185738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa9118-f334-4d0c-a765-c7d7df918291 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.190934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 220/444] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:52.191854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.215199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:52.220823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.221448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.221750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.222018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.223124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.224570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.226154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 225/445] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:52.227597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.307116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cb1f-9a98-4757-b624-0c6848ab81f3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.339682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:52.345297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8099d2-fb7f-4779-954c-94e0072ca78f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.347595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.348226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.348504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.348777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] PATH_INFO: `/v3/projects/f2c320d38a1546d0a6784ac68419376d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.349922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c320d38a1546d0a6784ac68419376d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.351342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.353004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 221/446] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/projects/f2c320d38a1546d0a6784ac68419376d => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:52.358029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.392091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.392734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.393020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.393295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.404736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:52.410161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:52.411269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:52.412109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 226/447] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:52.412882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.478072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3167dc-b103-4cdb-9026-9ab13fa80d39 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.490353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:52.514009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.514672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.514954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.515226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.516009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=2d1fb653600f44dda837472d2b0cfc75, scope.project.id=f2c320d38a1546d0a6784ac68419376d) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:52.516314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.517853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.527224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 222/448] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=2d1fb653600f44dda837472d2b0cfc75 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:52.527393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.542955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb229a-4ae0-419f-b475-358d993d4b9a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.589068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.589760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.590044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.590327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] PATH_INFO: `/v3/groups/b876cec3dc56414cbbce157e60155a70` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.592753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] RBAC: Authorizing `identity:get_group(group_id=b876cec3dc56414cbbce157e60155a70)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.594534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.596488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 227/449] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/groups/b876cec3dc56414cbbce157e60155a70 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:52.602596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.626544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:52.694666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05797bc9-7a68-4ad9-9c2b-1bdb8192df6e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.744085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebb05-5cd2-4b60-a5a0-6e869fd01416 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.750901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.751787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.752073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.752350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.753027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:52.753331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.754768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.766042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 223/450] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:52.771492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:52.782792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:52.793307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.798968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.798968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.798968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:52.798968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] RBAC: Adding query filter params (group.id=b876cec3dc56414cbbce157e60155a70, role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=f2c320d38a1546d0a6784ac68419376d) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:52.798968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:52.798968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:52.803625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 228/451] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/role_assignments?group.id=b876cec3dc56414cbbce157e60155a70&scope.project.id=f2c320d38a1546d0a6784ac68419376d&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:55:52.922778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb9b73-6b71-43f0-a725-fbfd6ef0a74f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:52.955814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:52.975724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:52.975724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:52.976027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:52.976255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:53.080791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:53.082314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:53.257610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 224/452] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:55:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:55:53.263863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:53.362483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfa8f7-6214-4f23-bfb0-8eab5bc8c6ee None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:53.395001 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:53.395619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:53.395903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:53.396189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] PATH_INFO: `/v3/projects/677669b803a2416fad6430932644a341` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:53.397293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] RBAC: Authorizing `identity:get_project(project_id=677669b803a2416fad6430932644a341)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:53.399342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:53.400947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 229/453] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:55:53 2022] GET /identity/v3/projects/677669b803a2416fad6430932644a341 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:53.406513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:53.426872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:53.503886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1c296-b886-4a58-ad42-f94485c43205 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:53.544166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597b2b5-1da9-49b6-95d4-f3213c6c59d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:53.544718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597b2b5-1da9-49b6-95d4-f3213c6c59d6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:53.545002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597b2b5-1da9-49b6-95d4-f3213c6c59d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:53.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597b2b5-1da9-49b6-95d4-f3213c6c59d6 None admin] PATH_INFO: `/v3/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:53.546426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597b2b5-1da9-49b6-95d4-f3213c6c59d6 None admin] RBAC: Authorizing `identity:get_group(group_id=cf9f0b3b4b08421f997c03d9a6cb6bbc)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:53.547881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597b2b5-1da9-49b6-95d4-f3213c6c59d6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:53.549435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 225/454] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Sep 4 15:55:53 2022] GET /identity/v3/groups/cf9f0b3b4b08421f997c03d9a6cb6bbc => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:53.555854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:53.613936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:53.656770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951ad3d-95f6-4ebc-97cb-8bf1cc57f79b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:53.689117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6d72-c3ba-4ee0-8dcd-56e5b03ab633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:53.689740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6d72-c3ba-4ee0-8dcd-56e5b03ab633 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:53.690026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6d72-c3ba-4ee0-8dcd-56e5b03ab633 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:53.690300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6d72-c3ba-4ee0-8dcd-56e5b03ab633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:53.691021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c6d72-c3ba-4ee0-8dcd-56e5b03ab633 None admin] RBAC: Adding query filter params (group.id=cf9f0b3b4b08421f997c03d9a6cb6bbc, role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=677669b803a2416fad6430932644a341) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:53.691330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c6d72-c3ba-4ee0-8dcd-56e5b03ab633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:53.692749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c6d72-c3ba-4ee0-8dcd-56e5b03ab633 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:53.699066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 230/455] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Sep 4 15:55:53 2022] GET /identity/v3/role_assignments?group.id=cf9f0b3b4b08421f997c03d9a6cb6bbc&scope.project.id=677669b803a2416fad6430932644a341&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:55.966077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149f111-317e-4831-a5ef-33af3cfe75c4 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:55.966932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149f111-317e-4831-a5ef-33af3cfe75c4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:55.967386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149f111-317e-4831-a5ef-33af3cfe75c4 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:55.969174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 226/456] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:55.974523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25c977-1fc8-455c-a705-035f8a71005f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:55.974948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25c977-1fc8-455c-a705-035f8a71005f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:55.975266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25c977-1fc8-455c-a705-035f8a71005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:55.998978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cb25c977-1fc8-455c-a705-035f8a71005f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:56.000177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cb25c977-1fc8-455c-a705-035f8a71005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:56.097132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 231/457] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:56.244917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248747-ff18-4324-bb5f-c010930cb087 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:56.245211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248747-ff18-4324-bb5f-c010930cb087 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:56.245441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248747-ff18-4324-bb5f-c010930cb087 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:56.247078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 227/458] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:56.252730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793912b5-61e3-4929-a0fb-6b8d0133b2aa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:56.253025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793912b5-61e3-4929-a0fb-6b8d0133b2aa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:56.253301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793912b5-61e3-4929-a0fb-6b8d0133b2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:56.278557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-793912b5-61e3-4929-a0fb-6b8d0133b2aa None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:56.279819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-793912b5-61e3-4929-a0fb-6b8d0133b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:56.358489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:56.383605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 232/459] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:56.546314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c248747-ff18-4324-bb5f-c010930cb087 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:56.572137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf5158-eca5-4ec4-a888-bb309b2718b5 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:56.572474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf5158-eca5-4ec4-a888-bb309b2718b5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:56.572704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf5158-eca5-4ec4-a888-bb309b2718b5 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:56.574564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 233/460] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:56.579318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83497d-d86d-4d6f-b961-8a0f04b59e75 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:56.579605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83497d-d86d-4d6f-b961-8a0f04b59e75 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:56.579883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83497d-d86d-4d6f-b961-8a0f04b59e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:56.598368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9b83497d-d86d-4d6f-b961-8a0f04b59e75 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:56.599552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9b83497d-d86d-4d6f-b961-8a0f04b59e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:56.703723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:56.705392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 234/461] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:56.718586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c248747-ff18-4324-bb5f-c010930cb087 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:56.770423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:56.771058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:56.771337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:56.771614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:56.787871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83497d-d86d-4d6f-b961-8a0f04b59e75 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:56.793984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:56.795445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:55:56.825375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:56.826347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:55:56.827186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:55:56.827186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:56.827186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:56.833703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 228/462] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:56.838968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:56.930356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83497d-d86d-4d6f-b961-8a0f04b59e75 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:56.943696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:56.953101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5455545-f929-495b-a1cd-ebdcee9a8394 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:56.966110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55c82b-87c4-4ab8-a824-da68c6199ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:56.967074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55c82b-87c4-4ab8-a824-da68c6199ccc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:56.967605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55c82b-87c4-4ab8-a824-da68c6199ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:56.973237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55c82b-87c4-4ab8-a824-da68c6199ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:56.980142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55c82b-87c4-4ab8-a824-da68c6199ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:56.981650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55c82b-87c4-4ab8-a824-da68c6199ccc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:56.994000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5722024-5f0f-4dbb-9310-8322f72299aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:56.995314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5722024-5f0f-4dbb-9310-8322f72299aa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:56.995801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5722024-5f0f-4dbb-9310-8322f72299aa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.001907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5722024-5f0f-4dbb-9310-8322f72299aa None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.001907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5722024-5f0f-4dbb-9310-8322f72299aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:57.001907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5722024-5f0f-4dbb-9310-8322f72299aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ca55c82b-87c4-4ab8-a824-da68c6199ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:55:57.002142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:57.003351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:55:57.004566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:55:57.004566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:57.004566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:57.004738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5722024-5f0f-4dbb-9310-8322f72299aa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.004807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 235/463] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:55:57.004807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda2bc5-1fab-4401-aea7-5fb013a710ae None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.004807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda2bc5-1fab-4401-aea7-5fb013a710ae None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.004807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda2bc5-1fab-4401-aea7-5fb013a710ae None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.006936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 236/464] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:57.008294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda2bc5-1fab-4401-aea7-5fb013a710ae None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.022750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:57.033202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 229/465] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:57.033202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6f7df-b8a3-483f-ae9c-40d61cdbfb8d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.033202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6f7df-b8a3-483f-ae9c-40d61cdbfb8d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.033202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6f7df-b8a3-483f-ae9c-40d61cdbfb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.052973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-65f6f7df-b8a3-483f-ae9c-40d61cdbfb8d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:57.054988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-65f6f7df-b8a3-483f-ae9c-40d61cdbfb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:57.133598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda2bc5-1fab-4401-aea7-5fb013a710ae None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:57.133925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 230/466] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:57.133925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6f7df-b8a3-483f-ae9c-40d61cdbfb8d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.175530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:57.176171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.176458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.177483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.178177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:57.178488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.181442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.199163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 237/467] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:57.200008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.232182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:57.265792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6f7df-b8a3-483f-ae9c-40d61cdbfb8d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:57.312287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:57.319270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:57.319954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.320996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.321276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.322180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.323901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.325886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5ba99-014c-4765-80de-fa9d435d8f65 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:57.364890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:57.365598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.365882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.366159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.377404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.379015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:55:57.394136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:57.395107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:55:57.399558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:55:57.399558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:57.399558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:57.399558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 238/468] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:57.399558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.421997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 231/469] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:57 2022] POST /identity/v3/users => generated 246 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:57.423015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.498049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:57.529967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230da35-8d09-4cc1-ae04-601133bd0f50 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:57.574001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f2674-ea16-4ee6-a177-5f6b1edc7e8a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:57.576941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4d3bf-566c-4e5e-9bba-d5b9f17fbd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:57.577867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4d3bf-566c-4e5e-9bba-d5b9f17fbd83 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.578505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4d3bf-566c-4e5e-9bba-d5b9f17fbd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.578996 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4d3bf-566c-4e5e-9bba-d5b9f17fbd83 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.579911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4d3bf-566c-4e5e-9bba-d5b9f17fbd83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.581554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4d3bf-566c-4e5e-9bba-d5b9f17fbd83 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.610519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:57.611122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.611410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.611675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.612326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:57.612606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.614005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.626708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 232/470] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:57.627508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.666680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 239/471] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:57 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:57.668164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d6fe6-7b7d-4fa4-afe5-c2aa138ce4f0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.668605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d6fe6-7b7d-4fa4-afe5-c2aa138ce4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.669020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d6fe6-7b7d-4fa4-afe5-c2aa138ce4f0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.670729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 240/472] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:55:57.671620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d6fe6-7b7d-4fa4-afe5-c2aa138ce4f0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.688217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:57.769323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c72d6b-9920-43d0-8033-88ced710ba14 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:57.818311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d6fe6-7b7d-4fa4-afe5-c2aa138ce4f0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:57.819041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043ecda-968d-44c8-88ea-9a4fa9a34832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:57.819740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ecda-968d-44c8-88ea-9a4fa9a34832 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.820101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ecda-968d-44c8-88ea-9a4fa9a34832 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.820501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ecda-968d-44c8-88ea-9a4fa9a34832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.834758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043ecda-968d-44c8-88ea-9a4fa9a34832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.836886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043ecda-968d-44c8-88ea-9a4fa9a34832 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7043ecda-968d-44c8-88ea-9a4fa9a34832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:55:57.853255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:57.854445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:55:57.855497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:55:57.855497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:55:57.855497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:57.856464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 233/473] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:57.858511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d039979-5cab-4a4b-bad0-e4c297b73733 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.858511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d039979-5cab-4a4b-bad0-e4c297b73733 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.858511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d039979-5cab-4a4b-bad0-e4c297b73733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.865872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:57.873018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:57.873909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:57.874020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:57.874713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:57.875000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:57.876700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:57.878309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2d039979-5cab-4a4b-bad0-e4c297b73733 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:55:57.878882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2d039979-5cab-4a4b-bad0-e4c297b73733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:55:57.958177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 241/474] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:57 2022] POST /identity/v3/users => generated 248 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:57.958177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:57.964600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 234/475] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:55:58.128671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd3336-52e0-4f12-b278-2fcc782afe6b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:58.177797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:58.178414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:58.178693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:58.178969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:58.179796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:58.180006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:58.181391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:58.197539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 242/476] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:55:58.203662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d039979-5cab-4a4b-bad0-e4c297b73733 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:58.245418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:58.317000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:58.322767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d039979-5cab-4a4b-bad0-e4c297b73733 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:58.364330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:58.365085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:58.365462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:58.365849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:58.366513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:58.368281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:58.391781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:58.449623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e8276-0478-4954-aecc-7aa2529140fb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:58.472410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 235/477] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:55:58 2022] POST /identity/v3/users => generated 249 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:55:58.496679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:58.497706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:58.498203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:58.498657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:58.499626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:58.501623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:58.551051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.sql.core [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 04 15:55:58.551051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 04 15:55:58.551051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [parameters: {'id': '93ba09fb8deb466fb0983da8b78f682e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 04 15:55:58.551051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81625) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Sep 04 15:55:58.569129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return ref.to_dict() Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 15:55:58.570838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application yield resource Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.rollback() Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.force_reraise() Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise self.value Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application session.commit() Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 04 15:55:58.571794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._prepare_impl() Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.flush() Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._flush(objects) Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application compat.raise_( Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application flush_context.execute() Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rec.execute(self) Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 04 15:55:58.572643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 15:55:58.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 15:55:58.574464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [parameters: {'id': '93ba09fb8deb466fb0983da8b78f682e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:58.575441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 04 15:55:58.576349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:55:58.579233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 243/478] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Sep 4 15:55:58 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:55:58.587841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:58.698093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:58.714733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc4ae4-84a3-4c21-92df-995f6959c54a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:58.751659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4a589-7c87-4cdf-8e9b-06c6fcf2be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:58.752697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4a589-7c87-4cdf-8e9b-06c6fcf2be09 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:58.753083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4a589-7c87-4cdf-8e9b-06c6fcf2be09 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:58.753445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4a589-7c87-4cdf-8e9b-06c6fcf2be09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:58.763900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4a589-7c87-4cdf-8e9b-06c6fcf2be09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-58a4a589-7c87-4cdf-8e9b-06c6fcf2be09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:55:58.768962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:55:58.770111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 15:55:58.771121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:55:58.777587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 236/479] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:55:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:55:58.781757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:55:58.964824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b8cd-d27e-4c80-a88c-eaf52781c60e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:55:58.987076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:55:59.023922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb3a8f-4eb2-4ff1-9ac5-29a7c8ac3183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:55:59.025185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3a8f-4eb2-4ff1-9ac5-29a7c8ac3183 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:55:59.025854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3a8f-4eb2-4ff1-9ac5-29a7c8ac3183 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:55:59.026576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3a8f-4eb2-4ff1-9ac5-29a7c8ac3183 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:55:59.027899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb3a8f-4eb2-4ff1-9ac5-29a7c8ac3183 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:55:59.028864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb3a8f-4eb2-4ff1-9ac5-29a7c8ac3183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:55:59.031701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb3a8f-4eb2-4ff1-9ac5-29a7c8ac3183 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:55:59.048735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 244/480] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:55:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:00.199751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3613206-1ef2-4dcf-a436-570a54bf5b84 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:00.200315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3613206-1ef2-4dcf-a436-570a54bf5b84 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:00.202142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebd0e-179d-4c9e-b648-323ba17790fb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:00.202524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebd0e-179d-4c9e-b648-323ba17790fb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:00.202859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebd0e-179d-4c9e-b648-323ba17790fb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:00.203242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3613206-1ef2-4dcf-a436-570a54bf5b84 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:00.206487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 245/481] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:00.206722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 237/482] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:00.209713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e493a-aab4-4922-a24f-f50e6ed85283 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:00.209998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e493a-aab4-4922-a24f-f50e6ed85283 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:00.210271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e493a-aab4-4922-a24f-f50e6ed85283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:00.217679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145244a-235d-4885-b115-327c3751fa69 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:00.217966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145244a-235d-4885-b115-327c3751fa69 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:00.218238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145244a-235d-4885-b115-327c3751fa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:00.231705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a73e493a-aab4-4922-a24f-f50e6ed85283 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:00.232884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a73e493a-aab4-4922-a24f-f50e6ed85283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:00.253270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8145244a-235d-4885-b115-327c3751fa69 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:00.254625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8145244a-235d-4885-b115-327c3751fa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:00.371647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 246/483] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:00.377668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 238/484] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:00.618306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e70353-2da3-4067-b56d-2f9d5792ee34 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:00.618698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e70353-2da3-4067-b56d-2f9d5792ee34 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:00.618876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e70353-2da3-4067-b56d-2f9d5792ee34 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:00.620530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 247/485] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:00.625773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b4ef-1ca7-434d-ae8f-9e87bc5def4c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:00.626325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b4ef-1ca7-434d-ae8f-9e87bc5def4c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:00.626600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b4ef-1ca7-434d-ae8f-9e87bc5def4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:00.648820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9db8b4ef-1ca7-434d-ae8f-9e87bc5def4c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:00.650141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9db8b4ef-1ca7-434d-ae8f-9e87bc5def4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:00.740544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 239/486] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:00.756191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e70353-2da3-4067-b56d-2f9d5792ee34 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:00.781270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b4ef-1ca7-434d-ae8f-9e87bc5def4c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:00.791384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:00.963493 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e70353-2da3-4067-b56d-2f9d5792ee34 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:00.966529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b4ef-1ca7-434d-ae8f-9e87bc5def4c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:00.969896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.028105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.028678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.028965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.029244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.030355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.031235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.031611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.031999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.042869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:01.043347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:01.048142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:01.049213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:01.050044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:01.050658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:01.051454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 240/487] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:01.052240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.055876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 248/488] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:01.056843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.123213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.162562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869203f8-333a-41a2-96e6-56b417bb952f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.193920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bc63f-a91f-4262-b9d7-5d0727c13094 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.200347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.201051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.201404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.201777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.217628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:01.223321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:01.224233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:01.225025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 241/489] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:01.225704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.246518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef2078-e053-4550-af5b-f411cc3cc49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.247141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef2078-e053-4550-af5b-f411cc3cc49f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.247576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef2078-e053-4550-af5b-f411cc3cc49f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.247922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef2078-e053-4550-af5b-f411cc3cc49f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.248666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef2078-e053-4550-af5b-f411cc3cc49f None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:01.249041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef2078-e053-4550-af5b-f411cc3cc49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.250551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef2078-e053-4550-af5b-f411cc3cc49f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.259268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 249/490] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:01.259268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8fee6-d399-46e6-884b-d176007e466b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.259268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8fee6-d399-46e6-884b-d176007e466b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.259268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8fee6-d399-46e6-884b-d176007e466b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.265701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 250/491] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:01.267034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8fee6-d399-46e6-884b-d176007e466b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.309147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.345103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647f334-4e54-4259-820a-ded17be0bee1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.369155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.379990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.380375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.380653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.380982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.381674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:01.381976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.384006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.392507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 242/492] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:01.393218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.401708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8fee6-d399-46e6-884b-d176007e466b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.443333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcab11-b417-4fae-88dc-0246e73b8343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.444091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcab11-b417-4fae-88dc-0246e73b8343 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.444513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcab11-b417-4fae-88dc-0246e73b8343 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.444795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcab11-b417-4fae-88dc-0246e73b8343 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.446344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcab11-b417-4fae-88dc-0246e73b8343 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:01.446737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcab11-b417-4fae-88dc-0246e73b8343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.448552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcab11-b417-4fae-88dc-0246e73b8343 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.464383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 251/493] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:01.465700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ba161-71ef-4d59-99a2-9847e7194abd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.465991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ba161-71ef-4d59-99a2-9847e7194abd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.466266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ba161-71ef-4d59-99a2-9847e7194abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.485873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1e4ba161-71ef-4d59-99a2-9847e7194abd None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:01.487003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1e4ba161-71ef-4d59-99a2-9847e7194abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:01.502734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adabf99-8eb9-46c4-8349-a3922200c8f7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.541687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.542334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.542624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.542883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.553924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.555576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.561776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 252/494] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:01.561776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ba161-71ef-4d59-99a2-9847e7194abd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:01.582571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:01.583530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:01.584536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:01.584536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:01.584536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:01.584536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 243/495] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:01.585135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.598646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.628664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.692785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53febd8b-9b31-4121-9ec3-aa389fe2a8ca None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.698099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ba161-71ef-4d59-99a2-9847e7194abd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.729247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.729901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.730177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.730444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.736810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75789fa-f465-462e-a5a9-e68b6f959740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.737470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75789fa-f465-462e-a5a9-e68b6f959740 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.737806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75789fa-f465-462e-a5a9-e68b6f959740 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.738082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75789fa-f465-462e-a5a9-e68b6f959740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.740315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.741800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.750086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75789fa-f465-462e-a5a9-e68b6f959740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.751930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75789fa-f465-462e-a5a9-e68b6f959740 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:01.756867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:01.757855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:01.758722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:01.758722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:01.758722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:01.761719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 244/496] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:01.763195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b75789fa-f465-462e-a5a9-e68b6f959740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:01.767819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:01.768680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:01.769507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:01.769507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:01.769507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:01.773899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 253/497] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:01.775752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2139f60-e12d-4949-b9ee-06f40d135469 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.776041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2139f60-e12d-4949-b9ee-06f40d135469 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.776318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2139f60-e12d-4949-b9ee-06f40d135469 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.777703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 254/498] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:01.782291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2139f60-e12d-4949-b9ee-06f40d135469 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.857392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.865157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:01.866535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8683b-12c2-48a7-abb7-d9490fbfcbba None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.898285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2139f60-e12d-4949-b9ee-06f40d135469 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:01.901242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.901873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.902149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.902416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.903049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:01.903368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.904720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.915830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 245/499] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:01.916615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:01.934436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7e3de-d742-4393-a46c-b44a7131225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:01.935041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7e3de-d742-4393-a46c-b44a7131225e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.935344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7e3de-d742-4393-a46c-b44a7131225e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.935614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7e3de-d742-4393-a46c-b44a7131225e None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.936247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7e3de-d742-4393-a46c-b44a7131225e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:01.936547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7e3de-d742-4393-a46c-b44a7131225e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:01.937970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7e3de-d742-4393-a46c-b44a7131225e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:01.950093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 255/500] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:01.951293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d4f3-7fe1-4d2b-9608-b6a1e5513263 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:01.951581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d4f3-7fe1-4d2b-9608-b6a1e5513263 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:01.952522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d4f3-7fe1-4d2b-9608-b6a1e5513263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:01.968859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-faf9d4f3-7fe1-4d2b-9608-b6a1e5513263 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:01.970093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-faf9d4f3-7fe1-4d2b-9608-b6a1e5513263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:02.016230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daadbf3-9dcc-4160-943b-19f8d1291a5c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.034796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 256/501] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:02.035618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9d4f3-7fe1-4d2b-9608-b6a1e5513263 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.049763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.050353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.050631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.050900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.051532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:02.051828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.053174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.064882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 246/502] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:02.065888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.078384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.107821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.151142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9d4f3-7fe1-4d2b-9608-b6a1e5513263 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.173486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcf7a6-684e-407a-afba-6c21fdcc576c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.191163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.191985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.192437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.192852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.203787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:02.209157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:02.210229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:02.211816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:02.212553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.212553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.212553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.212553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.214505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 257/503] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:02.215440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.224894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.226403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:02.239270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:02.240362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 247/504] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:02.241256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.319656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.336606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8a844-a310-42ed-b81e-970f7bb46027 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.352012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57a1e4-e800-48ae-8382-b00b35cc0b74 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.357110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.371967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.372600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.372882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.373164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.380599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.382210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.387600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.388226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.388503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.388776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:02.400300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:02.401734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 258/505] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:02.402623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.404814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.406954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:02.420236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:02.421178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:02.422141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 248/506] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:02.422721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.518852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767b768-0f5c-413f-bab4-6296a88748a3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.521419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9c06-5e6e-46c2-aab8-6eb72b0eb443 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.549809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.553360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.553999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.554280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.554668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.555233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:02.555535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.555623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.556145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.556427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.556702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.556913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.557428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:02.557766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.564190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.564696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 259/507] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:02.565481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.596323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 249/508] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:02.597166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.609212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.666046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523b8e2-9aff-46a1-a987-1d1236670845 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.698810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.699435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.699714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.699983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.700148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e7d1f-5cc7-4616-9b00-3e3e09b276eb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.710217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.712331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:02.726105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:02.727293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:02.728199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:02.728199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:02.728199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:02.728199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 260/509] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:02.728407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.739077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.739786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.740080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.740361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.741109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:02.741420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.744624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.768574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 250/510] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:02.768574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.822498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.826941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:02.833297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5bb47-20c3-4bf4-8bc7-86e2ebaccec5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.866140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.866682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.867505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.867793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.868502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:02.868806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.870277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.875599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b70ba-20c4-4fe9-a8a0-70348412a322 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:02.906154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 261/511] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:02.906983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:02.909298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:02.909963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:02.910292 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:02.910611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:02.923352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:02.924792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:02.939312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:02.940196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:02.941041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:02.941041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:02.941041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:02.941041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 251/512] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:02.941257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.008875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e597f1-6e74-4772-9982-8dee36a27bda None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.039456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.042123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.042734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.043020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.043299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.043393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251b0fa-d398-4db3-a981-71e2573eb37d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.072610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.079560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.080170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.080447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.080716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.081406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:03.081928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.083681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.096015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 252/513] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:03.096680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.122701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.124142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.194230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d9423-7815-4d7b-a9c4-32ad79b4395a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.222601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 262/514] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:03.223439 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.228088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.228693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.228974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.229245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.258440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.274744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.297706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.299164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.319707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175001bc-b4a6-4d61-b86d-23dd55a2b955 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.355682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.356326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.356563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.356836 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.396928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 253/515] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:03.397763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.427020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.428509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.464315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.480183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.506877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca6904-5522-4d18-93fc-f01f3132d823 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.529268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 263/516] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:03.530124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.540154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.540765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.541047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.541317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.541997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:03.542302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.543675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.555846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 254/517] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:03.556633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.637011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33e2f-29f1-4e1a-a11e-346e1f69813d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.656997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1289f2e-e423-4f55-8efd-295a0d1a8b78 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.669939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.670541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.670817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.671095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.671736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.673172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.674416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.691083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.691712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.692093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.692270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.702592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.704054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:03.718949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:03.720014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:03.720983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:03.720983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:03.720983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:03.720983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 255/518] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:03.720983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.782849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 264/519] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:56:03 2022] POST /identity/v3/users => generated 247 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:03.783801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.799564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.825934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc42fe-1b30-4da1-99a1-511370341f99 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.866582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.868491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.868491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.868653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.880795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.882295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:03.899920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:03.901062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:03.901949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:03.901949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:03.901949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:03.901949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 256/520] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:03.902769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:03.918899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f518cda-a22a-4fd6-b3c3-4e110ac7119e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:03.934065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:03.958055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:03.958700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:03.958983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:03.959262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:03.973298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:03.975505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:03.990939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:03.992689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:03.993537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:03.993537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:03.993537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:03.993537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 265/521] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:03.993919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.024792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dadf9-e114-499e-bf58-7f0408b12f4a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.060986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.061169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.061715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.062041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.062387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.070479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.074583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:04.085088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:04.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:04.087868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 257/522] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:04.088415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.113747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c8f7-fbcd-441b-8bbc-e0144ecb7f21 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.153473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.154130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.154412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.154689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.155330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:04.155637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.157007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.169917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 266/523] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:04.170684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.182920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.202857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97b6e-1343-4cee-a0f5-1fe68f33f4a9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.240329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.242285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.242795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.243268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.244092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:04.244600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.246015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.264696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 258/524] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:04.265441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.281477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c94f4b-8d31-4827-bf1a-fbd77fe6bfc5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.303434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.324176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.324810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.325098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.325379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.326508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:04.326822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.328649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.343285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 267/525] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:04.348403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.379314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c512-444b-49e5-ba5d-ccbffb14f8be None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.418186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.418857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.419137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.419402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.420196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:04.420442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.421888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.436609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.440070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 259/526] 10.222.0.39 () {56 vars in 1139 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:04.440940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.465175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f7365-5c72-4336-b168-e63769930933 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.500356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.501297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.501630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.501902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.509516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.510977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:04.522887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:04.524412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:04.525268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 268/527] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:04.525813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.550552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9b5d3-7264-4e30-9c2a-d87572c8f528 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.568215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.584469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.585125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.585403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.586260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.592895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.594352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:04.606401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:04.607254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:04.608095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 260/528] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:04.608572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.635616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b481c7-565d-4ed8-9715-5c5b901e8f5d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.670003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.670684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.670989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.671188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.672143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.678058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.680042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:04.691734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:04.692907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 269/529] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:04.693775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.718849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17ccff-7b16-4194-875c-7daae949292f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.756302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.756914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.757194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.757626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.801868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.806109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b2294-9f2e-44ea-84ce-8866545bfcaa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:04.830465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.831917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.842307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:04.842934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:04.843221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:04.843504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:04.844315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:04.844628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:04.846222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:04.870190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 270/530] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:04.870190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.894509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:04.945290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 261/531] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:04.945290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:04.981212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63c966-1a50-47bb-ba7b-2d4c8af157af None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.017463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.018102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.018388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.018667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.019358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:05.019671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.021108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.040334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 271/532] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:05.040334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:05.052320 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdafb9-9d6c-46cb-a751-4ad9ff2d4b76 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.056857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:05.093435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.094186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.094462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.094741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.095429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:05.095733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.097149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.115352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 262/533] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:05.116163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:05.139654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:05.142176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368e456-48ad-413c-b252-e43e16bf5f27 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.178150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.179406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.179761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.180035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.180753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=46fd540df66143b2b768cc8d9f4beed7) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:05.181058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.182758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.197722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 272/534] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:05 2022] GET /identity/v3/role_assignments?user.id=46fd540df66143b2b768cc8d9f4beed7&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:05.197722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:05.240483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636f27a-f5b4-4816-9f15-e07ddae5da77 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.291338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.292582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.292982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.293356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.304427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:05.310712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.313049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.332021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a385aa-c4c6-4b28-9180-df2424728511 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:05.340113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:05.341205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:05.342966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:05.342966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:05.342966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:05.349871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 263/535] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:05.349871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:05.388199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.388953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.389240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.389523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.390295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=c029d59023b04bf8b8bf9d4d5eee7390) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:05.390613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.392310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.406104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 273/536] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:05 2022] GET /identity/v3/role_assignments?user.id=c029d59023b04bf8b8bf9d4d5eee7390&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:05.407157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:05.434023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:05.471762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fb180-de7e-425c-a3f3-cb444689f0ef None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.568980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a227d-5cf7-46bd-9674-e6888dc8f6c1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.578328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.578966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.579248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.579523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.580283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=c95204bd338a4a99a510a67c4ab97532) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:05.580600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.582095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.600888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 264/537] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:05 2022] GET /identity/v3/role_assignments?user.id=c95204bd338a4a99a510a67c4ab97532&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:05.606879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a550b5-2887-4c39-800b-9c915b431ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.608174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a550b5-2887-4c39-800b-9c915b431ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.608174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a550b5-2887-4c39-800b-9c915b431ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.608174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a550b5-2887-4c39-800b-9c915b431ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.609244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a550b5-2887-4c39-800b-9c915b431ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:05.609946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a550b5-2887-4c39-800b-9c915b431ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.612649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a550b5-2887-4c39-800b-9c915b431ad2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.630633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 274/538] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:05.637486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:05.768368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbd695-c4a2-4416-a36a-9f910c5a7f3f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:05.806223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45117f-318c-4bc0-922b-854353447d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:05.806862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45117f-318c-4bc0-922b-854353447d91 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.807138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45117f-318c-4bc0-922b-854353447d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.807408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45117f-318c-4bc0-922b-854353447d91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.810437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:05.815173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45117f-318c-4bc0-922b-854353447d91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:05.816709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45117f-318c-4bc0-922b-854353447d91 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0e45117f-318c-4bc0-922b-854353447d91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:05.829047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:05.830896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:05.832978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:05.833963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848f177-4fd6-47f6-a43f-c3918ffcfbdd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.833963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848f177-4fd6-47f6-a43f-c3918ffcfbdd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.833963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848f177-4fd6-47f6-a43f-c3918ffcfbdd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.834404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 265/539] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:05.834544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 275/540] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:05.840334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848f177-4fd6-47f6-a43f-c3918ffcfbdd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:05.840680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be18e6-912b-4b91-9067-cb01d59f6bcf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:05.840971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be18e6-912b-4b91-9067-cb01d59f6bcf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:05.841249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be18e6-912b-4b91-9067-cb01d59f6bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:05.855050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:05.863850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a6be18e6-912b-4b91-9067-cb01d59f6bcf None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:05.869163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a6be18e6-912b-4b91-9067-cb01d59f6bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:05.942071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 266/541] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:06.042346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848f177-4fd6-47f6-a43f-c3918ffcfbdd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:06.089942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58136-de45-42bb-9301-977313509573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:06.090561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58136-de45-42bb-9301-977313509573 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:06.090846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58136-de45-42bb-9301-977313509573 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:06.091123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58136-de45-42bb-9301-977313509573 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:06.091832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb58136-de45-42bb-9301-977313509573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:06.092142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb58136-de45-42bb-9301-977313509573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:06.093603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb58136-de45-42bb-9301-977313509573 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:06.114767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 276/542] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:06.121428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be18e6-912b-4b91-9067-cb01d59f6bcf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:06.276060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be18e6-912b-4b91-9067-cb01d59f6bcf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:06.320636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:06.332636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:06.333658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:06.333658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:06.333810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:06.334541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=93ef027007184b8180325d6fb26324a9) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:06.334851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:06.336479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:06.345731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 267/543] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:06 2022] GET /identity/v3/role_assignments?user.id=93ef027007184b8180325d6fb26324a9&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:06.364700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58136-de45-42bb-9301-977313509573 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:06.464059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:06.516095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58136-de45-42bb-9301-977313509573 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:06.582872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:06.583616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:06.583807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:06.584087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:06.603927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:06.609332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:06.610249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:06.611058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:06.614861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 277/544] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:06.621261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:06.750788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1bd2c-0e2e-489f-848f-83eec9b81470 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:06.799448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:06.800117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:06.800400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:06.800669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:06.801342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:06.801669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:06.803035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:06.817016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 268/545] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:06.822859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:06.941836 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e758e5-aa88-4f29-bd67-c78ba1dc9c0a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:06.969821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:06.983518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:06.988968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:06.988968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:06.988968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:06.996955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:06.998438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:07.025047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:07.026057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:07.026968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:07.026968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:07.026968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:07.026968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 278/546] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:07.031846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:07.112654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:07.184950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99befc-64cb-4acb-a70f-193a671f029c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:07.225809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:07.226420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.226704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.226976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.227621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:07.227921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:07.229294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:07.248821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 269/547] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:07.254195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:07.405242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677aa5e-60f1-465a-9fdc-5f3c34d43de9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:07.450741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:07.451380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.451658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.451928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.459612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:07.461119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:07.473094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:07.474251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:07.475157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 279/548] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:07.480389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:07.611721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda2ea-4427-451d-94ed-2242b3b3092f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:07.626541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:07.662496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974a7fd-de37-4b93-9345-e97bd6786641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:07.663105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974a7fd-de37-4b93-9345-e97bd6786641 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.663382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974a7fd-de37-4b93-9345-e97bd6786641 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.663789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974a7fd-de37-4b93-9345-e97bd6786641 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.664897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974a7fd-de37-4b93-9345-e97bd6786641 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:07.665202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974a7fd-de37-4b93-9345-e97bd6786641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:07.666651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974a7fd-de37-4b93-9345-e97bd6786641 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:07.692363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 270/549] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Sep 4 15:56:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:07.698065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:07.726814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea2736-f507-46c4-acc7-5015da9e0d06 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.727440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea2736-f507-46c4-acc7-5015da9e0d06 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.727943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea2736-f507-46c4-acc7-5015da9e0d06 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.729920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 271/550] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:07.735968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13817143-d463-4b72-bda2-1ed3900c500e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.735968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13817143-d463-4b72-bda2-1ed3900c500e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.736290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13817143-d463-4b72-bda2-1ed3900c500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.755185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:07.756265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-13817143-d463-4b72-bda2-1ed3900c500e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:07.757816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-13817143-d463-4b72-bda2-1ed3900c500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:07.813206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db154c85-e169-4081-ab44-c92fd00568f8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:07.828560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 272/551] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:56:07.842232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae9b61-5a70-46c1-b2ce-64031d9f78ef None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.842676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae9b61-5a70-46c1-b2ce-64031d9f78ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.843063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae9b61-5a70-46c1-b2ce-64031d9f78ef None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.844830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 273/552] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:07 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:07.849804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed2310-fcf9-484b-b5db-8d3c759df13f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.850199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed2310-fcf9-484b-b5db-8d3c759df13f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.850579 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed2310-fcf9-484b-b5db-8d3c759df13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.851314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26cd25-9152-4ff3-9345-82ad81c50d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:07.852092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26cd25-9152-4ff3-9345-82ad81c50d19 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:07.852512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26cd25-9152-4ff3-9345-82ad81c50d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:07.852941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26cd25-9152-4ff3-9345-82ad81c50d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:07.890738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-81ed2310-fcf9-484b-b5db-8d3c759df13f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:07.892116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-81ed2310-fcf9-484b-b5db-8d3c759df13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:07.928474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26cd25-9152-4ff3-9345-82ad81c50d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:07.931135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26cd25-9152-4ff3-9345-82ad81c50d19 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:07.976984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:07.987184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 274/553] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:56:08.046055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 280/554] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:08.054711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed2310-fcf9-484b-b5db-8d3c759df13f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.094403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:08.149823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28c241-9cd1-4988-bc37-f4293d1e3c4b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.150182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28c241-9cd1-4988-bc37-f4293d1e3c4b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.150388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28c241-9cd1-4988-bc37-f4293d1e3c4b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.152017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 281/555] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:08.157746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85e8c-41d5-4761-b2ed-261e17f566d2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.158040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85e8c-41d5-4761-b2ed-261e17f566d2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.158322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85e8c-41d5-4761-b2ed-261e17f566d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.179815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-27e85e8c-41d5-4761-b2ed-261e17f566d2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:08.181210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-27e85e8c-41d5-4761-b2ed-261e17f566d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:08.200957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed2310-fcf9-484b-b5db-8d3c759df13f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.268216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.268772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.269046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.269312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.285248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:08.287040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:08.296139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 282/556] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:08.296982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85e8c-41d5-4761-b2ed-261e17f566d2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:08.307417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:08.308578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:08.309694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:08.309694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:08.309694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:08.309694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 275/557] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:08.312600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.387690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:08.423341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:08.424969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85e8c-41d5-4761-b2ed-261e17f566d2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.451295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908ded8-31e4-420b-82ff-a8df446fa8d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.468431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.469447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.469692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.470210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.482152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:08.487264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:08.488319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:08.489161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 283/558] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:08.489902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.495917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.497089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.497540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.497987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.510099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:08.516195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:08.517717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:08.518606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:08.525833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 276/559] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:08.527004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.598268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f69da-fb68-4ec7-96e3-1c8974d166f0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.628389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:08.634298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.639882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.639882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.639882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.639882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:08.639882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:08.639882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:08.640578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37b3f3-3064-4c7f-b979-2b683c05de51 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.651051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 284/560] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:08.652396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.682413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.683029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.683312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.683588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.684250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:08.684554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:08.685959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:08.694307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 277/561] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:08.695311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.701208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:08.766761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21a3a8-16ad-4471-8713-0934128708bb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.803042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.803572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.803747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b868a-405f-442a-a7fb-094ce6c29675 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.803850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.804147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.804815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:08.805107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:08.807002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:08.819183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 285/562] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:08.819183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.845585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.846319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.846672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.847015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.861164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:08.866489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:08.867992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:08.868852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 278/563] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:08.870111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:08.875226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:08.923035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba2b17-58f5-414a-b212-0c3714727a6d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.930173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:08.956337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:08.956975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:08.957260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:08.957537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:08.964270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:08.965753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:08.969116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18bb07-7142-4c13-8f11-1efab81eb731 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:08.977591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:08.978500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:08.979388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 286/564] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:08.979864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.004376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15480d-05f7-41b2-a938-bb0ed84f3200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.005003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15480d-05f7-41b2-a938-bb0ed84f3200 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.005276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15480d-05f7-41b2-a938-bb0ed84f3200 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.005548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15480d-05f7-41b2-a938-bb0ed84f3200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.016252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15480d-05f7-41b2-a938-bb0ed84f3200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.017696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15480d-05f7-41b2-a938-bb0ed84f3200 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-aa15480d-05f7-41b2-a938-bb0ed84f3200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:09.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:09.032652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:09.034592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:09.034592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:09.034592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:09.034592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 279/565] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:09.034592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a677-fd18-47e0-bb4e-c96d045889a5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.034592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a677-fd18-47e0-bb4e-c96d045889a5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.034592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a677-fd18-47e0-bb4e-c96d045889a5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.037604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 280/566] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:09.037604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a677-fd18-47e0-bb4e-c96d045889a5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.085158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015dd27-6273-4494-a4c9-11b480c8e694 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.097912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.119537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.120162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.120442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.120714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.131825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.134431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.137620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a677-fd18-47e0-bb4e-c96d045889a5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:09.148979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:09.150009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:09.150886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:09.150886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:09.150886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:09.150886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 287/567] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:09.150886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.158462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.176981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.177612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.177940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.178215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.178887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:09.179734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.181107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.188912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 281/568] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:09.189831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.255884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd17044-9ca1-4834-be94-0d2126eb09b7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.290570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8a4bb-6329-4070-a3e0-815b0b5383e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.291176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a4bb-6329-4070-a3e0-815b0b5383e1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.291451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a4bb-6329-4070-a3e0-815b0b5383e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.291723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a4bb-6329-4070-a3e0-815b0b5383e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.292400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8a4bb-6329-4070-a3e0-815b0b5383e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:09.292702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8a4bb-6329-4070-a3e0-815b0b5383e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.294162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8a4bb-6329-4070-a3e0-815b0b5383e1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.294448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766eecc-a543-4eb3-98d0-aff99fb857cf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.305633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 288/569] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:09.306862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66000948-25c3-4bf0-9d38-4902f52f507b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.307142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66000948-25c3-4bf0-9d38-4902f52f507b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.307419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66000948-25c3-4bf0-9d38-4902f52f507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.326416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-66000948-25c3-4bf0-9d38-4902f52f507b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:09.327627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-66000948-25c3-4bf0-9d38-4902f52f507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:09.335952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.336731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.337015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.337288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.337971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:09.338270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.339636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.345468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.351988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 282/570] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:09.352791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.392155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.394852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 289/571] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:09.395681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66000948-25c3-4bf0-9d38-4902f52f507b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.472213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cde777-a521-4d14-8633-dac7de6048b2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.501526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66000948-25c3-4bf0-9d38-4902f52f507b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.510043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.510654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.510934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.511210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.511849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:09.512154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.513513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.529715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 283/572] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:09.529715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.541225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.541977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.542345 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.542620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.552979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.554521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:09.569644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:09.571197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:09.572164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:09.572164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:09.572164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:09.576628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 290/573] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:09.577371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.588237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.640958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.645906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa778de-860e-4413-900c-e142915d7324 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.682041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.682652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.682936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.683212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.683947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=91bc1f3a0e9046bab7aa2c799ad46c3b) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:09.684257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.685696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.686505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b5cf-7123-45d9-b9ed-a033049c57fa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.691039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 284/574] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/role_assignments?user.id=91bc1f3a0e9046bab7aa2c799ad46c3b&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:09.691820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.738110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.738766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.739050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.739324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.748938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.750518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:09.763973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:09.765023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:09.766005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 291/575] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:09.766645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.828528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f515200-efe0-4750-8b9f-1abcb77695de None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.850133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.865775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.866415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.867328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.867602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.875534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.877236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.886703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a580cf-89cd-4473-b287-192e14532cde None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:09.888598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:09.889481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:09.890463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:09.894287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 285/576] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:09.895129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:09.920497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:09.926933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:09.927549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:09.927832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:09.928113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:09.928802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:09.929672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:09.934664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:09.948039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 292/577] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:09.948039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.004426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc577f-f06d-4ddd-86be-0b0e77f7e75d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.039892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.040524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.040810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.041086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.051180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:10.056899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:10.058045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 286/578] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:10.058886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.065969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee18ab9-33cf-4df2-9297-16abaabdb7b0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.101655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.112754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.113460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.113874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.114162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.114900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:10.115210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.117204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.136514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 293/579] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:56:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:10.138519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.148919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.166704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4b7ae-957e-4c88-8f65-fa0c8faa1245 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.202900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.203512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.203800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.204794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:10.205152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.206644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.224338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 287/580] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:10.226500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.248314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b5fc2-7e10-4533-b044-f70b164fd225 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.284991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.285756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.286043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.286324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.294125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.295567 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:10.307276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:10.308322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 294/581] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:10.309244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.346772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce436d-0e2e-4e32-b5b3-7d560c9b9c77 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.369079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.383594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.384206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.384489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.384771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.385416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:10.385750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.387165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.397965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 288/582] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:10.399275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.403140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.428322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c20758-d9ed-4150-8fcf-99f86de2d498 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.468840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.469482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.469792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.470062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.481895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.483368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:10.497934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:10.498863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:10.499760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:10.499760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:10.499760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:10.499760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 295/583] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:10.499760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.521314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbbbbd-323c-4a3c-ac35-5ae4a3fe2b44 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.558127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.558753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.559044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.559321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.571091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.572545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:10.588106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:10.593946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:10.595081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:10.595081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:10.595081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:10.595081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 289/584] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:10.595081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.619535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.624786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648c03-9ca8-4142-934f-60f39150ec16 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.651555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.663262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.666374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.666374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.666374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.666374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:10.666374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.667887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.687142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 296/585] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:10.688002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.702375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704df0-ed4f-4b05-91c3-6c0f97bca3bc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.740227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.741207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.742511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.743028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.756192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.757902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:10.774895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:10.776075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:10.777245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:10.777245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:10.777245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:10.777245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 290/586] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:10.777245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.811749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43992b-fefb-4bd0-8af1-ffadf70d0f4f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.846474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.847085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.847373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.847635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.848272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:10.848574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.849956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.861312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 297/587] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:10.862057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.878929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.883566 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d776c17-e9b9-48c8-b30e-3371cef0a978 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:10.890804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:10.920707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:10.921428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:10.921841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:10.922174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:10.922894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:10.923250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:10.924853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:10.937795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 291/588] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:10.938742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:10.966529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c563df-15f6-4293-adbf-db7ffce18b99 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.005603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.006333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.006618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.006892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.017498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.018992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:11.034383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:11.035348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:11.037010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:11.037010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:11.037010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:11.037010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 298/589] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:11.037010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.050791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0b085-f46e-4d54-a50f-7d1419553617 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.086703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.087332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.087586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.087863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.088505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:11.088811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.090201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.103102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 292/590] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:11.103735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.111030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.133436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.151338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d24d42-0fbe-4c51-b645-39a136fe0002 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.189179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.189833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.190114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.190385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.197513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.198976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:11.211266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:11.212179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:11.218307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 299/591] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:11.218907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a990-dfac-4e90-b3ca-bbe188317db7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.219003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.252982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.253644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.253919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.254189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.261746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.263168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:11.274973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:11.280647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 293/592] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:11.281862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.341667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000028e-13cb-4215-af2d-db5e6107ed22 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.367745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.367923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.378000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.378610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.378889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.381512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.381512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:11.381512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.381512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.386540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5f2c4-14ce-4dcd-8261-863060ed32a5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.398884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 300/593] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:11.398884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.422552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.423180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.423566 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.423838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.432166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.434499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:11.446229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:11.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 294/594] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:11.448355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.506277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcfd57-120b-403e-b788-cd78d87ee607 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.540600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.541206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.541482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.541861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.542488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:11.542790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.544228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.554738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce9fc0-e73d-4191-8eda-571508a9eeb0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.561348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 301/595] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:11.561348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.596208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.596807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.597085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.597474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.598198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:11.598506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.599961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.608343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.617440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 295/596] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:11.617440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.626919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.670709 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0889b-4791-4b74-aca5-76d2ff983eca None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.705789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.706890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.706890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.707281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.714567 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.716014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.724054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c265-a5d0-4a6e-aa8e-b983112ac788 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:11.728160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:11.729164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:11.730176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 302/597] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:11.730650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.757814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.758422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.758700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.758977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.759693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:11.759994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.761448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.778946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 296/598] 10.222.0.39 () {56 vars in 1139 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:11.779725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:11.832602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc1f8-c067-4170-9c49-ee9005b14e11 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.840372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.842986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:11.866336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b855a-51f8-4e19-a40d-123e069a264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.867611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b855a-51f8-4e19-a40d-123e069a264d None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.867898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b855a-51f8-4e19-a40d-123e069a264d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.868179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b855a-51f8-4e19-a40d-123e069a264d None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/46fd540df66143b2b768cc8d9f4beed7/roles/fa94d66c831d468f83e224236c343293` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.879831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efef31-0bd4-4191-8a52-39e93ea85550 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:11.887541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b855a-51f8-4e19-a40d-123e069a264d None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=46fd540df66143b2b768cc8d9f4beed7, role_id=fa94d66c831d468f83e224236c343293)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.887541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b855a-51f8-4e19-a40d-123e069a264d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.913923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:11.914585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.914865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.915139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/c029d59023b04bf8b8bf9d4d5eee7390/roles/fa94d66c831d468f83e224236c343293` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.930076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=c029d59023b04bf8b8bf9d4d5eee7390, role_id=fa94d66c831d468f83e224236c343293)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:11.930387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:11.978798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 303/599] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:11 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/46fd540df66143b2b768cc8d9f4beed7/roles/fa94d66c831d468f83e224236c343293 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:56:11.980142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77186240-5674-4040-8361-b4473a05dee0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:11.980425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77186240-5674-4040-8361-b4473a05dee0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:11.980698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77186240-5674-4040-8361-b4473a05dee0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:11.982028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 304/600] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:11.982738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77186240-5674-4040-8361-b4473a05dee0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.025858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 297/601] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:11 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/c029d59023b04bf8b8bf9d4d5eee7390/roles/fa94d66c831d468f83e224236c343293 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:56:12.027186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.131503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77186240-5674-4040-8361-b4473a05dee0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:12.182632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bb687-4002-4e45-bae2-8fb62c19dead None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:12.187835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed7c0d-3622-4a50-8164-0a92a233f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:12.188454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7c0d-3622-4a50-8164-0a92a233f154 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:12.188735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7c0d-3622-4a50-8164-0a92a233f154 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:12.189006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7c0d-3622-4a50-8164-0a92a233f154 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:12.189789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed7c0d-3622-4a50-8164-0a92a233f154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:12.190104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed7c0d-3622-4a50-8164-0a92a233f154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:12.192122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed7c0d-3622-4a50-8164-0a92a233f154 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:12.201201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:12.208026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 305/602] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:12.209155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72186a1-cc91-4280-b829-20ae56cae8ca None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:12.209622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72186a1-cc91-4280-b829-20ae56cae8ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:12.209947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72186a1-cc91-4280-b829-20ae56cae8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:12.226508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:12.227256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:12.227600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:12.227937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:12.232213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f72186a1-cc91-4280-b829-20ae56cae8ca None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:12.233517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f72186a1-cc91-4280-b829-20ae56cae8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:12.239707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:12.241353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:12.245840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:12.262684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:12.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:12.265275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:12.265275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:12.265275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:12.265275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 298/603] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:12.265275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.333601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 306/604] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:12.333601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72186a1-cc91-4280-b829-20ae56cae8ca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.402354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740976-0ff6-443a-a94d-b7ae473f2c90 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:12.442864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:12.443448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:12.443725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:12.444239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/c95204bd338a4a99a510a67c4ab97532/roles/fa94d66c831d468f83e224236c343293` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:12.475510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=c95204bd338a4a99a510a67c4ab97532, role_id=fa94d66c831d468f83e224236c343293)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:12.477664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:12.523876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72186a1-cc91-4280-b829-20ae56cae8ca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:12.558727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:12.559338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:12.559613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:12.559884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:12.560522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:12.560820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:12.562216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:12.574770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 307/605] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:12.579797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.592266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:12.596267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 299/606] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:12 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/c95204bd338a4a99a510a67c4ab97532/roles/fa94d66c831d468f83e224236c343293 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:56:12.674046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.707611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:12.733271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee131cf-cc27-4d8f-8957-f40f0b4984a4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:12.809046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:12.809967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:12.810334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:12.810751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:12.825066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:12.826782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:12.845377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:12.846598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:12.847737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 308/607] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:12.848331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73797fbb-15e7-45ae-b5fe-b315601dfda5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:12.856749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.889990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:12.890622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:12.890896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:12.891166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:12.912772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:12.918795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:12.919925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:12.921019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 300/608] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:12.925715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:12.994559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:13.057600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a2399-ca67-40a1-a8c9-02a4a64c397c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:13.086063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:13.097094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:13.097761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:13.102087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:13.102087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:13.102087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:13.102087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:13.102087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:13.121488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93978f-9fd4-4dda-9a41-0115f9a6c421 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:13.123915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 309/609] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:13.129997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:13.162127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:13.162804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:13.163148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:13.163544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:13.164748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:13.165062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:13.166536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:13.175543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 301/610] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:13.182239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:13.291779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783a6bd-b026-46a6-aa70-9dfe6adbcae9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:13.323387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:13.345184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:13.346140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:13.346572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:13.346997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/93ef027007184b8180325d6fb26324a9/roles/fa94d66c831d468f83e224236c343293` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:13.378005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=93ef027007184b8180325d6fb26324a9, role_id=fa94d66c831d468f83e224236c343293)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:13.379736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:13.414842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768c737-4729-48d1-841d-c410a800d30f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:13.449911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:13.450755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:13.451196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:13.451634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:13.462822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:13.465764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 310/611] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:13 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/93ef027007184b8180325d6fb26324a9/roles/fa94d66c831d468f83e224236c343293 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 15:56:13.469632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:13.471296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:13.488017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:13.489297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:13.490286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:13.490286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:13.490286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:13.491035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 302/612] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:13.496768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:13.683038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9106d4-b6a7-423d-a58d-66bb6259c705 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:13.727522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:13.728144 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:13.728428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:13.728705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:13.729466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:13.729820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:13.731197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:13.745622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 311/613] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:13.751659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:13.892410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3ece9-bec5-4523-b862-6536dd603cc5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:13.938365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:13.938935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:13.939223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:13.939495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:13.947580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:13.949017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:13.965676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:13.966647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:13.967576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 303/614] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:13.973837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:14.046027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:14.146437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de34a2-5180-446c-bd0c-858e805c17c9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:14.194026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b84294-44b5-4f6f-a965-3207d4123a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:14.195553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b84294-44b5-4f6f-a965-3207d4123a65 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.196047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b84294-44b5-4f6f-a965-3207d4123a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.196547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b84294-44b5-4f6f-a965-3207d4123a65 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.197497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b84294-44b5-4f6f-a965-3207d4123a65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:14.198037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b84294-44b5-4f6f-a965-3207d4123a65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:14.199887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b84294-44b5-4f6f-a965-3207d4123a65 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:14.226438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 312/615] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Sep 4 15:56:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:14.234678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:14.330738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:14.358313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ef839-f3e0-4b15-ac57-611dbb1d5850 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.358938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ef839-f3e0-4b15-ac57-611dbb1d5850 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.359419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ef839-f3e0-4b15-ac57-611dbb1d5850 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.361355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 313/616] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:14.366102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0ce9-5f57-43c2-8625-bc88f9aafea4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.366713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0ce9-5f57-43c2-8625-bc88f9aafea4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.367266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0ce9-5f57-43c2-8625-bc88f9aafea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.392190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-814e0ce9-5f57-43c2-8625-bc88f9aafea4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:14.393903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-814e0ce9-5f57-43c2-8625-bc88f9aafea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:14.398003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84c0cc-34e1-4e4a-8c3e-b54ed316c448 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:14.435728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02386619-03ac-4b98-9723-a745baae13d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:14.436370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02386619-03ac-4b98-9723-a745baae13d3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.436650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02386619-03ac-4b98-9723-a745baae13d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.436925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02386619-03ac-4b98-9723-a745baae13d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.448829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02386619-03ac-4b98-9723-a745baae13d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:14.450232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02386619-03ac-4b98-9723-a745baae13d3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-02386619-03ac-4b98-9723-a745baae13d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:14.466260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:14.467219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:14.468284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:14.468284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:14.468284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:14.468284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 304/617] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:14.468999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d5af4-8bdc-4729-a97f-516089e442c5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.469178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d5af4-8bdc-4729-a97f-516089e442c5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.469411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d5af4-8bdc-4729-a97f-516089e442c5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.470902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 305/618] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:14.473741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 314/619] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:14.476698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0ce9-5f57-43c2-8625-bc88f9aafea4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:14.477519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875521fc-b8f3-496b-ae8c-384271071ef8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.478726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875521fc-b8f3-496b-ae8c-384271071ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.479330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875521fc-b8f3-496b-ae8c-384271071ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.530621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-875521fc-b8f3-496b-ae8c-384271071ef8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:14.531785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-875521fc-b8f3-496b-ae8c-384271071ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:14.532084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:14.591784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0ce9-5f57-43c2-8625-bc88f9aafea4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:14.631716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:14.632947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.633440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.634061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.634915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:14.635441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:14.637021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:14.650037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 315/620] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:14.658589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:14.692547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 306/621] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:56:14.754334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d386-4998-4f67-9e7f-bc240850fd60 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.754942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d386-4998-4f67-9e7f-bc240850fd60 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.754942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d386-4998-4f67-9e7f-bc240850fd60 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.756168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 307/622] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:14.761346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd7bf-f869-47bd-a04e-c36f133b109e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.761662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd7bf-f869-47bd-a04e-c36f133b109e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.761943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd7bf-f869-47bd-a04e-c36f133b109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.787204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-13ccd7bf-f869-47bd-a04e-c36f133b109e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:14.788855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-13ccd7bf-f869-47bd-a04e-c36f133b109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:14.791514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:14.809247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f57a9-7f60-4151-9839-922fd735ecc0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:14.815821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:14.856708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:14.857701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:14.858215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:14.858663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:14.867939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:14.869678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:14.878042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 308/623] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:56:14.879032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd7bf-f869-47bd-a04e-c36f133b109e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:14.883952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:14.884864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:14.885883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:14.888570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 316/624] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:14.894526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.057966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd7bf-f869-47bd-a04e-c36f133b109e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.061700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514156d6-9a35-4257-8707-3d2ef1479162 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.103094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.103812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.104162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.104540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.105101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.105988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.106606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.107081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.108252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:15.108746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:15.110659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:15.116088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:15.122214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:15.123170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:15.124015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:15.127475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 309/625] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:15.128469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.135868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 317/626] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:15.142034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.147850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:15.160061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:15.244578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566d6dd-6403-4b5f-b39b-69266dfd3a5a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.257280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063ed2c-9fa9-4254-84fd-07e60940575c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.279140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.279783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.280078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.280324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.290834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:15.292918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.293529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.293842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.294116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.294786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:15.295094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:15.295655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:15.296552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:15.297434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:15.298490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:15.301525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 310/627] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:15.302646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.307118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 318/628] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:15.307801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.386891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:15.401189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:15.419443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93045-66fb-4c59-8f09-236ce7151ec1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.422566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852c941-bf42-4440-a9ca-a41c48dccfc0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.457175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.457497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.458007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.458103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.458300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.458398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.458579 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/91bc1f3a0e9046bab7aa2c799ad46c3b/roles/fa94d66c831d468f83e224236c343293` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.458677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.469763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:15.474804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=91bc1f3a0e9046bab7aa2c799ad46c3b, role_id=fa94d66c831d468f83e224236c343293)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:15.475112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:15.476213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 319/629] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:15.477716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.481423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:15.552455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 311/630] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:15 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/91bc1f3a0e9046bab7aa2c799ad46c3b/roles/fa94d66c831d468f83e224236c343293 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:56:15.553316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.594062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a50b6-3428-4da0-8b0f-42d032511eaa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.627028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:15.637828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.642161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.642161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.642161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.642161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:15.642161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:15.642161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:15.657618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 320/631] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:15.657618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.710001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25ac9f-52da-423d-81ca-4519536e953f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.752275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:15.758309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.758985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.759205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.759523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.771786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:15.774072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:15.789193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:15.790211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:15.791104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:15.791104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:15.791104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:15.793444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 312/632] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:15.794512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.796413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523684-a3c5-40cd-b301-39b228f0477d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.845461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.846053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.846505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.846787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.847464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:15.848042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:15.849434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:15.861437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 321/633] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:15.866147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:15.910168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44b103-5a0e-4485-aaaf-0a7502104b08 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.926012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:15.947255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79f82d-7a49-4b3f-b7f1-4b3127a48a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:15.948130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79f82d-7a49-4b3f-b7f1-4b3127a48a78 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.948604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79f82d-7a49-4b3f-b7f1-4b3127a48a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.949054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79f82d-7a49-4b3f-b7f1-4b3127a48a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.961712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79f82d-7a49-4b3f-b7f1-4b3127a48a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:15.963277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79f82d-7a49-4b3f-b7f1-4b3127a48a78 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-dd79f82d-7a49-4b3f-b7f1-4b3127a48a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:15.979246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:15.980448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:15.981327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:15.981327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:15.981327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:15.981327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 313/634] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:15.981833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a4e7c-b7ef-4174-8824-9a1626955592 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:15.982115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a4e7c-b7ef-4174-8824-9a1626955592 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:15.982390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a4e7c-b7ef-4174-8824-9a1626955592 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:15.983643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb830d-aea7-47eb-b18c-c9e8f71d5ab9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:15.983853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 314/635] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:15.984570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a4e7c-b7ef-4174-8824-9a1626955592 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.025754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:16.027925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.028547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.028836 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.029116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.029793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:16.030369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.031488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.044570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 322/636] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:16.045407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.098794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a4e7c-b7ef-4174-8824-9a1626955592 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.133689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fc466-2030-4f50-82b3-ca43a9986dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.134324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fc466-2030-4f50-82b3-ca43a9986dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.134605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fc466-2030-4f50-82b3-ca43a9986dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.134884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fc466-2030-4f50-82b3-ca43a9986dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.144773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fc466-2030-4f50-82b3-ca43a9986dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.146247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fc466-2030-4f50-82b3-ca43a9986dc0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.158692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb9527-8132-4ec8-98b9-439f4732f275 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-658fc466-2030-4f50-82b3-ca43a9986dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:16.163875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:16.165585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:16.166484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:16.166484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:16.166484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:16.166484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 315/637] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:16.166484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575f51d-7d67-4730-b9de-be04f7eef146 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.166484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575f51d-7d67-4730-b9de-be04f7eef146 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.166843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575f51d-7d67-4730-b9de-be04f7eef146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.181340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:16.187462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6575f51d-7d67-4730-b9de-be04f7eef146 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:16.188672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6575f51d-7d67-4730-b9de-be04f7eef146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:16.206639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.207255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.207537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.207812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.208585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:16.208890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.211071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.229058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 323/638] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:16.229058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.256570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 316/639] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:16.262519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575f51d-7d67-4730-b9de-be04f7eef146 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.292778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:16.357784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef2440-2b10-4b10-acc8-fc28c6147ee9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.383385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575f51d-7d67-4730-b9de-be04f7eef146 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.396703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.397756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.398479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.398964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.407325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.409107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.420095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.420832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.421173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.421523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.422278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:16.422641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:16.422997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:16.423941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:16.424922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:16.425464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.427760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 324/640] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:16.428789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.441803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 317/641] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:16.441803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.467587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:16.528717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:16.545960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e30239-b57d-4dbc-bf58-f959efbf8767 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.552404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcb6e-0afd-4cd4-97aa-efe6f62d2781 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.582645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.583280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.583566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.583845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.590593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.591264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.591584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.591855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.592513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.592615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:16.592876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.594022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.594311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:16.606370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:16.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 325/642] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:16.610608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.622928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 318/643] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:16.623739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.714049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:16.725082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04301587-a0fa-473f-a7eb-c7cabdb2966c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.731924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1849c-8354-4f98-ba8b-8a4fa1eb765f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.765530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.766211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.766560 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.766853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.773833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.774810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.775207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.775668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.776116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.776247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.779230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:16.787242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:16.788195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:16.804538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 326/644] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:16.805903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:16.806737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:16.807689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 319/645] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:16.808707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.902016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4972d-a0e8-4966-840f-3b64ef4e1475 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.910349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8035028-4418-4222-855a-236f5a258a93 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:16.939986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.940800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.941101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.941381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.942148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:16.942458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:16.944137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:16.948180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:16.948796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:16.949062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:16.949333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:16.967515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 327/646] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:16.967515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:16.977333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.018300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.026550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.028023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.070349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfffea-7a74-4535-a9e5-b375e559cfbd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.106692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.107303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.107591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.107867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.108603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:17.108915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.110385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.133025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 328/647] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:17.133025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.137870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 320/648] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:17.142342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.208551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.248583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b138b-4e3a-4e38-afd2-7d457f6af61e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.250507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.258528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469d9bd-4463-4da3-b1fa-36ac9355c174 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.286003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.286974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.287258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.287532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.288208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:17.288898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.290429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.295951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.296611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.296825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.297096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.298337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 329/649] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:17.299124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.377257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.379155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.412245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e4e34-cf4d-41ea-aecd-a2937771ba2d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.441427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.450993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.454785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.454785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.454785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.484722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.491174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 321/650] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:17.493113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.526205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.527675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.601412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033bc68-742a-4b7f-9dd2-5706f4f55739 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.631812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 330/651] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:17.633343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.639946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.640586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.640867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.641142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.651287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.652719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.659755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:17.666962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:17.668650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:17.669883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:17.669883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.669883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:17.669883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 322/652] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:17.669883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.719629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.735203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af94399-2700-4095-b7d8-d60a721a18d1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.770806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fe7c6-0dd6-46a8-8d12-4a8d2f64cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.771458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fe7c6-0dd6-46a8-8d12-4a8d2f64cf0d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.771738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fe7c6-0dd6-46a8-8d12-4a8d2f64cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.772011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fe7c6-0dd6-46a8-8d12-4a8d2f64cf0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.773612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7d7c3-b98f-4124-a041-cf7cd9fb637f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.783768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65fe7c6-0dd6-46a8-8d12-4a8d2f64cf0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.785302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65fe7c6-0dd6-46a8-8d12-4a8d2f64cf0d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b65fe7c6-0dd6-46a8-8d12-4a8d2f64cf0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:17.803835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:17.805238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:17.806365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:17.806365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.806365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:17.806365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 331/653] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:17.807751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9fee3-922f-44dc-8dd1-176dcfa6a685 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.808041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9fee3-922f-44dc-8dd1-176dcfa6a685 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.808320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9fee3-922f-44dc-8dd1-176dcfa6a685 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.813610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 332/654] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:17.813610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9fee3-922f-44dc-8dd1-176dcfa6a685 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.816854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.817496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.817807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.818082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.829850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.831290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:17.846563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:17.847620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:17.848507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:17.848507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.848507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:17.856276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 323/655] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:17.856276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:17.902216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.922386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9fee3-922f-44dc-8dd1-176dcfa6a685 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.958206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:17.958863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:17.959142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:17.959420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:17.967439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:17.969824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:17.971221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47620c83-e493-41a0-bf61-1e74ec3143ae None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:17.971341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:17.986532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:17.987694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:17.988806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:17.988806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:17.988806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:17.988806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 333/656] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:17.988806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.010596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe9322-6b5e-424f-a7da-4de64951f1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.011339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe9322-6b5e-424f-a7da-4de64951f1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.011627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe9322-6b5e-424f-a7da-4de64951f1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.011902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe9322-6b5e-424f-a7da-4de64951f1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.012546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe9322-6b5e-424f-a7da-4de64951f1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:18.012845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe9322-6b5e-424f-a7da-4de64951f1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.014234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe9322-6b5e-424f-a7da-4de64951f1d3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.025706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 324/657] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:18.027123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7a946-7272-46c8-85d6-51ebd467dc99 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.027414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7a946-7272-46c8-85d6-51ebd467dc99 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.027694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7a946-7272-46c8-85d6-51ebd467dc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.045792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c0f7a946-7272-46c8-85d6-51ebd467dc99 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:18.046983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c0f7a946-7272-46c8-85d6-51ebd467dc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:18.089330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7dca9-1379-467c-9e1d-52609bc80946 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.121880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 325/658] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:18.121880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7a946-7272-46c8-85d6-51ebd467dc99 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.123213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.126280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.126888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.127164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.127434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.128070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:18.128394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.129759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.141799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 334/659] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:18.142878 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.227630 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.237204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7a946-7272-46c8-85d6-51ebd467dc99 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.268845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72958c29-f82b-4997-af3c-0c041cf1746d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.274757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.275368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.275674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.275921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.276565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:18.276863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.278254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.294865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 326/660] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:18.294865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.308140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.308732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.309012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.309290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.309962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:18.310268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.311629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.326300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 335/661] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:18.326955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.382929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.410796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e506450-4f1d-4040-a9e2-144df894ea74 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.429719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ee31-471d-4796-bcb9-9a3eebd50e7f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.444104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.444859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.445145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.445413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.452184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.453366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.462656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.463277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.463554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.463822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:18.465183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:18.466230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 327/662] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:18.472051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.472621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.473082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.484338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:18.484658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:18.485642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:18.486679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 336/663] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:18.487353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.578831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8b831-09c4-4509-a6f2-d80b2747e1cf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.597020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a81497-207a-471c-807e-ea5f298a2eeb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.609727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.614789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.615427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.615710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.615984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.622690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.625792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.631048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.631672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.632446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.632713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:18.637958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:18.639011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 328/664] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:18.640058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.644384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.646246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:18.658704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:18.659738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 337/665] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:18.660652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.714017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.744356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f86e5-67bc-4e36-be9b-b99611d3e80f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.761103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b498f-e09c-4861-8ffc-8c087c0bc6ba None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.779099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.779726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.780011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.780288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.790658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:18.794268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.794873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.795156 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.795430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:18.796089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:18.797025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:18.797919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:18.798582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:18.798582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.798722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 329/666] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:18.798791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.798868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.813920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 338/667] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:18.814732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.844655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.905166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95fa35-991a-4b2e-bcca-d7f47919416e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.917609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60988065-3a8f-43f7-9746-ac0c05f6e63d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:18.933453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:18.940118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.940783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.941070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.941342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.942102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:18.942407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.943865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.952467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:18.953154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:18.953521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:18.953823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:18.954511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:18.954842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:18.956281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:18.961592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 330/668] 10.222.0.39 () {56 vars in 1139 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:18.962425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:18.974230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 339/669] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:18.974230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.065450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.067996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f1934-907e-43ba-b621-1e30291a8833 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.075032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a0b49-6d47-40da-b42e-fcfc93f6c867 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.102744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.103357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.103639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.103913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.104607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:19.104910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.106370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.108723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.109330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.109640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.109918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.110580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:19.110886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.112260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.118602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 331/670] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:19.119192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.121263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 340/671] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:19.122229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.172004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.221991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c089-051f-48ba-8245-0f4511f2f480 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.223474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115d97a-b8c1-4ed3-8cf5-0419c1c59777 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.256015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.256553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.256665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.256910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.257745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.257948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.258040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.258285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.259009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=46fd540df66143b2b768cc8d9f4beed7) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:19.259315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.260729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.266394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 332/672] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/role_assignments?user.id=46fd540df66143b2b768cc8d9f4beed7&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:19.267222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.283118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.341284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.342811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.392995 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d02cbc-71fd-4193-b2a3-1df410bcac87 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.407239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.430607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.431219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.431502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.432454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.432558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=c029d59023b04bf8b8bf9d4d5eee7390) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:19.432801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.434319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.442512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 333/673] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/role_assignments?user.id=c029d59023b04bf8b8bf9d4d5eee7390&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:19.443468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.464221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 341/674] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:19.465318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.550438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.578656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bbcc2-beeb-4826-a63d-3a4ddf667d57 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.605816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838f767-88d0-489f-9a51-3713c10be71e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.626234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.626851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.627133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.627407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.629024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=c95204bd338a4a99a510a67c4ab97532) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:19.629339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.631202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.638222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 334/675] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/role_assignments?user.id=c95204bd338a4a99a510a67c4ab97532&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:19.638866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.652943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.653927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.654585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.654771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.668385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.670115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:19.689132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:19.690415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:19.691409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:19.691409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:19.691409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:19.692050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 342/676] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:19.693593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.698475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.766702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7b433-5539-4d73-8e86-d154aba145b3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.806931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.807875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.808368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.808824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.820011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e000-5dc4-4775-9306-0c1a275c1761 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:19.824038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.825765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.828196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:19.842651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:19.843644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:19.844532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:19.844532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:19.844532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:19.844532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 335/677] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:19.850833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.870672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:19.871403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:19.871700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:19.871984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:19.872701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:19.873015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:19.875435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:19.894500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 343/678] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:19.900178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:19.979345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:19.992725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562aee0-02d8-4a2a-b748-55763af79761 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.015456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20114b2d-19d8-44dd-84c4-d7452643e66b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.038957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.040309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.041072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.041538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:20.042556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:20.043040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:20.045152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:20.054529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.055395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.055608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.055966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:20.061908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 336/679] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:20.066839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:20.067736 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:20.068620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:20.082512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:20.083503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:20.084454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 344/680] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:20.090938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:20.135212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:20.200633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778faad-ba04-441b-a508-670935404ff2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.221195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a004c74-163e-413f-9c31-2b2db43cc527 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.245104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.245777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.246063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.246338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:20.257467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:20.259092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:20.263228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.263848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.264127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.264400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:20.265135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:20.265441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:20.266884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:20.271364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:20.272283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:20.273678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 337/681] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:20.280340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:20.298010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:20.298806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 345/682] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Sep 4 15:56:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:20.306778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:20.410024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:20.449826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c6936-dd13-4c9c-993f-1e4d0904e604 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.463050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177a922-e9e1-4c22-8d47-0e9932cd0e7e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.521726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.522804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.523519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.523970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:20.530034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.530858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.531987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.532448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:20.533394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:20.540892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:20.544181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:20.562928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 338/683] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:20.570708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:20.654225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:20.655698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:20.698005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:20.748448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5a5a5-3aca-4744-8770-8b53db2657a6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.775949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:20.797218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 346/684] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 15:56:20.798503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:20.826884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.827498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.827778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.828043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:20.828862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=93ef027007184b8180325d6fb26324a9) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:20.829166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:20.843154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:20.856894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 339/685] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:20 2022] GET /identity/v3/role_assignments?user.id=93ef027007184b8180325d6fb26324a9&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:20.934141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f49dad-c2c1-46e2-97c7-874c3163ca1c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:20.987885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:20.988477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:20.988756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:20.989030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.004544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:21.006068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:21.023494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:21.024520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:21.025408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:21.025408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:21.025408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:21.025408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 347/686] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:21.030429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:21.139079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460e14b-117b-4b59-b7e3-eb72bb8409a8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:21.175283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:21.175902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.176186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.176456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.177101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:21.177399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:21.179369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:21.182017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:21.196561 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 340/687] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:21.201078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:21.267892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:21.309185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d35a76-b9df-4cce-bdb3-194d76a48e7d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:21.344636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:21.345303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.345613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.345894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.353057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:21.354545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:21.366905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:21.367937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:21.368830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:21.372858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 348/688] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:21.374255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:21.506005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a9a0d-ec97-4240-ace0-9233742fac11 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:21.554961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdd4e5-f35d-4cef-9091-26d7c64e456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:21.555499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdd4e5-f35d-4cef-9091-26d7c64e456a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.555774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdd4e5-f35d-4cef-9091-26d7c64e456a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.556044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdd4e5-f35d-4cef-9091-26d7c64e456a None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.556745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdd4e5-f35d-4cef-9091-26d7c64e456a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:21.557048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdd4e5-f35d-4cef-9091-26d7c64e456a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:21.558489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdd4e5-f35d-4cef-9091-26d7c64e456a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:21.582989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 341/689] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:21.589041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:21.671952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57b62f-86b1-4264-aff8-df8836dc7595 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.672300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57b62f-86b1-4264-aff8-df8836dc7595 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.672545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57b62f-86b1-4264-aff8-df8836dc7595 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.674203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 342/690] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:21.679186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5e9f2-07f1-4c14-83c6-ca635773a0de None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.679477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5e9f2-07f1-4c14-83c6-ca635773a0de None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.679859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5e9f2-07f1-4c14-83c6-ca635773a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.691036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:21.697283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dd3f8-3ed9-442a-a44b-8573085b7119 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:21.698983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d5c5e9f2-07f1-4c14-83c6-ca635773a0de None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:21.700204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d5c5e9f2-07f1-4c14-83c6-ca635773a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:21.736767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992b235-ff4a-4c40-bdbf-0879cb44dd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:21.737388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992b235-ff4a-4c40-bdbf-0879cb44dd59 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.737702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992b235-ff4a-4c40-bdbf-0879cb44dd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.737984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992b235-ff4a-4c40-bdbf-0879cb44dd59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.738782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992b235-ff4a-4c40-bdbf-0879cb44dd59 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=91bc1f3a0e9046bab7aa2c799ad46c3b) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:21.739092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992b235-ff4a-4c40-bdbf-0879cb44dd59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:21.740595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992b235-ff4a-4c40-bdbf-0879cb44dd59 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:21.753736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 349/691] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:21 2022] GET /identity/v3/role_assignments?user.id=91bc1f3a0e9046bab7aa2c799ad46c3b&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:21.814109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:21.842464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 343/692] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:56:21.970137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fae283-d261-47d7-b9c6-705e88ee03b2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.971122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fae283-d261-47d7-b9c6-705e88ee03b2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.971609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fae283-d261-47d7-b9c6-705e88ee03b2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:21.973394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 350/693] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:21.979744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487a83e-2e49-4f0f-b9cb-dd6292730cfe None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:21.980335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487a83e-2e49-4f0f-b9cb-dd6292730cfe None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:21.980819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487a83e-2e49-4f0f-b9cb-dd6292730cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.000156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0487a83e-2e49-4f0f-b9cb-dd6292730cfe None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:22.001944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0487a83e-2e49-4f0f-b9cb-dd6292730cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:22.070031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 344/694] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:22.164214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fae283-d261-47d7-b9c6-705e88ee03b2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:22.262276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a363a-90df-4781-89e6-9ed71a7e62fc None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:22.262631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a363a-90df-4781-89e6-9ed71a7e62fc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:22.262840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a363a-90df-4781-89e6-9ed71a7e62fc None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.265511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 345/695] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:22.270271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7eab3-c419-4194-9fd4-d2ee529c9cd9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:22.270873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7eab3-c419-4194-9fd4-d2ee529c9cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:22.271358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7eab3-c419-4194-9fd4-d2ee529c9cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.281351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fae283-d261-47d7-b9c6-705e88ee03b2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:22.297884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-29f7eab3-c419-4194-9fd4-d2ee529c9cd9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:22.299386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-29f7eab3-c419-4194-9fd4-d2ee529c9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:22.320755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:22.321465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:22.321846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:22.322196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.336831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:22.342838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:22.343740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:22.344571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 351/696] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:22.350378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:22.375090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:22.394938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 346/697] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:56:22.396538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7eab3-c419-4194-9fd4-d2ee529c9cd9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:22.457903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:22.508826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4340cc7-ff13-49b5-a148-f0a3795aebb8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:22.539730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7eab3-c419-4194-9fd4-d2ee529c9cd9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:22.590671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:22.591323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:22.591606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:22.591909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.593628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:22.594241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:22.594518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:22.594789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.595064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:22.596439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:22.597431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:22.597767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:22.599164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:22.618262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 352/698] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:22.621857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:22.622867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:22.624110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 347/699] 10.222.0.39 () {56 vars in 1129 bytes} [Sun Sep 4 15:56:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:22.625491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:22.629869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:22.785966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:22.788761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7146a2-c83c-4f4a-ba6f-cd9fde039527 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:22.832027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:22.833909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:22.834662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:22.835684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.849148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:22.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:22.870174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:22.871104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:22.871988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:22.871988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:22.871988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:22.874237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 353/700] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:22.875507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:22.916898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d21b3-fa0f-4669-b4f9-8fefb72fcf2e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:22.931619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:22.967198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:22.967967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:22.968379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:22.968794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:22.969506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:22.969940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:22.971463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:22.978015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 348/701] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Sep 4 15:56:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:22.979016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:22.994073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249abee1-1428-4c85-8bfc-5271eece98dc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.033903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.034772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.035238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.035677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.047577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:23.049271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:23.064634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:23.065705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:23.066607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:23.066607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:23.066607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:23.068808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 354/702] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:23.068963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:23.070585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:23.088922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274be8d-7e56-4134-af35-186c14ba62a1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.126350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.126978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.127416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.127658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.128388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:23.128921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:23.130295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:23.143409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 349/703] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:23.144436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:23.181775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:23.186384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445c528-e10a-452d-9596-5af16bd00f5a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.229205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.230324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.230704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.231057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.231788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:23.232431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:23.234319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:23.242249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 355/704] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Sep 4 15:56:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:23.243688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:23.259549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94c605-935e-42b2-bd9f-423f64dcf730 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.311326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.311964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.313206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.313450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.314127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:23.314433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:23.316960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:23.343081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 350/705] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:23.348297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:23.361987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:23.452234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624d26-d303-4acc-a9f4-b6b82a67d7ab None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.489473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:23.495917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8423df-ff16-4b2b-8120-62877e05e94f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.502317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.502952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.508437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.508437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.518763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:23.520274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:23.533728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd19a8-9211-4808-90b5-a138245108be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.534342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd19a8-9211-4808-90b5-a138245108be None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.534622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd19a8-9211-4808-90b5-a138245108be None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.534898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd19a8-9211-4808-90b5-a138245108be None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.535546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd19a8-9211-4808-90b5-a138245108be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:23.536970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd19a8-9211-4808-90b5-a138245108be None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:23.543374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:23.544329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:23.545277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:23.549192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 356/706] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:56:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:23.555898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:23.659126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 351/707] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:56:23 2022] POST /identity/v3/users => generated 254 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:23.660385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c754ba-416f-43f0-9616-d5018085b96f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.660683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c754ba-416f-43f0-9616-d5018085b96f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.660949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c754ba-416f-43f0-9616-d5018085b96f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.670627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 352/708] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:23 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:23.675609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebadde7-cf54-4694-a193-0713ca6a3277 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.679777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b78b1-67d2-4378-b09e-9c025026ae09 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.680132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b78b1-67d2-4378-b09e-9c025026ae09 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.680404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b78b1-67d2-4378-b09e-9c025026ae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.688573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:23.719825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.720555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.720848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.721128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:23.721979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:23.722347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:23.723869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:23.730681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ca5b78b1-67d2-4378-b09e-9c025026ae09 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:23.731915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca5b78b1-67d2-4378-b09e-9c025026ae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:23.747916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 357/709] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:56:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:23.753853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:23.855080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 353/710] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 15:56:23.918752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee1426-23fc-44bf-9992-3ded80d32cd7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:23.955966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:23.956927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:23.957413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:23.957916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.002121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:24.029342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:24.030844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:24.133544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 358/711] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:24.138850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b78b1-67d2-4378-b09e-9c025026ae09 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:24.155738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:24.243986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:24.249280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:24.275692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b78b1-67d2-4378-b09e-9c025026ae09 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:24.281372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01fef6-c241-4bf2-baa7-aa0cca5f6deb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:24.318180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:24.318991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.319122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:24.319514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.319881 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.320504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.320698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:24.320804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.321036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.322384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:24.333677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:24.335482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:24.337046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:24.337898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 359/712] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:24 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:24.343936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:24.345601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:24.347313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:24.363487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:24.364479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:24.365497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:24.365497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:24.365497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:24.369701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 354/713] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:24.371233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:24.482700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a69efd-e807-4d55-92c6-3d8263b34e74 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:24.549700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc4a93-f393-4d6b-ad6b-e13e0cf03320 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:24.553911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:24.562129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:24.562865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.563293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.563661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.569095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] RBAC: Adding query filter params (name=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:24.569095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:24.569095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:24.580148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 360/714] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:24.585316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:24.586600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:24.596627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:24.596627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.596627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.596959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.598480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:24.602627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:24.604922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:24.628110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 355/715] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:24.633312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:24.744876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc82729-1241-4a83-8318-785dfc4e6024 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:24.756643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ae414-e389-411d-9d38-f439ea025e13 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:24.792302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816936e2-3a43-4b10-9f0e-91c5a5709ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:24.793302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:24.793591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816936e2-3a43-4b10-9f0e-91c5a5709ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.793974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.794256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.794551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816936e2-3a43-4b10-9f0e-91c5a5709ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.794653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.795225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816936e2-3a43-4b10-9f0e-91c5a5709ed9 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.796096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816936e2-3a43-4b10-9f0e-91c5a5709ed9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:24.796717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816936e2-3a43-4b10-9f0e-91c5a5709ed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:24.798584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816936e2-3a43-4b10-9f0e-91c5a5709ed9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:24.803992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 361/716] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:24.804355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:24.805343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da20a4-1612-4ac9-bb53-bd1812206ab2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.805717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da20a4-1612-4ac9-bb53-bd1812206ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.805999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da20a4-1612-4ac9-bb53-bd1812206ab2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.806749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:24.811662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 362/717] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:24.813195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e79c6-e0d5-4be1-b644-cfe746043f87 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:24.813496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e79c6-e0d5-4be1-b644-cfe746043f87 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:24.813801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e79c6-e0d5-4be1-b644-cfe746043f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:24.823087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:24.824060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:24.824938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:24.827368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 356/718] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:24.834846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:24.836262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-212e79c6-e0d5-4be1-b644-cfe746043f87 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:24.837541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-212e79c6-e0d5-4be1-b644-cfe746043f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:24.976532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:24.981916 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:25.003409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 363/719] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:25.031492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7013-56b5-4485-b924-b71dc5ad8c13 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:25.082804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:25.083423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:25.083702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:25.083971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:25.084659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:25.085486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:25.086991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:25.103717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 357/720] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:25.110156 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e79c6-e0d5-4be1-b644-cfe746043f87 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:25.255797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e79c6-e0d5-4be1-b644-cfe746043f87 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:25.292323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:25.292935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:25.293218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:25.293494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:25.294256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=46fd540df66143b2b768cc8d9f4beed7) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:25.294618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:25.296054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:25.301967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 364/721] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:25 2022] GET /identity/v3/role_assignments?user.id=46fd540df66143b2b768cc8d9f4beed7&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:25.349305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:25.469483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7595-c22e-4df8-8a2a-1da06054a062 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:25.487765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:25.517269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf59dc-4327-4a39-a88a-adb37a89bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:25.517953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf59dc-4327-4a39-a88a-adb37a89bd15 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:25.518239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf59dc-4327-4a39-a88a-adb37a89bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:25.518508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf59dc-4327-4a39-a88a-adb37a89bd15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:25.535136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf59dc-4327-4a39-a88a-adb37a89bd15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b5bf59dc-4327-4a39-a88a-adb37a89bd15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:25.541167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:25.542125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:25.542956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 358/722] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:25.549472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:25.628570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:25.688581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657c924-a6f3-4e82-af0f-42e2c37ea2bc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:25.703495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893776b-db46-4d67-9909-70411094ff7c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:25.704058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893776b-db46-4d67-9909-70411094ff7c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:25.704479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893776b-db46-4d67-9909-70411094ff7c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:25.706036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 359/723] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:25.712298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dfb48-810f-4a09-8166-f70bbc2e344a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:25.712763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dfb48-810f-4a09-8166-f70bbc2e344a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:25.713183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dfb48-810f-4a09-8166-f70bbc2e344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:25.744392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-094dfb48-810f-4a09-8166-f70bbc2e344a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:25.746216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-094dfb48-810f-4a09-8166-f70bbc2e344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:25.763626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:25.764506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:25.766640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:25.767174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:25.768025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:25.769675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:25.771204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:25.793631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 365/724] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:25.801621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:25.888997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 360/725] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:25.943986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f88094-913d-4162-adf2-7fd60d2c721b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:25.966943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:25.995034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:25.995676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:25.995952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:25.996222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.014863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:26.016321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:26.041758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:26.042793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:26.043733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:26.043733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:26.043733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:26.043927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 366/726] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:26.048452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dfb48-810f-4a09-8166-f70bbc2e344a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:26.193372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dfb48-810f-4a09-8166-f70bbc2e344a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:26.196370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:26.240010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:26.241272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:26.241804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:26.242282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.243150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:26.243678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:26.245255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:26.249577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:26.269502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 361/727] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:26.274850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:26.368152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee0e73-24a4-46d1-8d65-a4b80b9431f9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:26.392640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207aa5-1271-488b-ab33-2298ef6f2500 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:26.407381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784216c-7442-4739-b56b-2c727f2b285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:26.408353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784216c-7442-4739-b56b-2c727f2b285d None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:26.408855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784216c-7442-4739-b56b-2c727f2b285d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:26.409329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784216c-7442-4739-b56b-2c727f2b285d None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.410232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784216c-7442-4739-b56b-2c727f2b285d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:26.411812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784216c-7442-4739-b56b-2c727f2b285d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:26.430075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac1681-9b49-44b8-967c-a22bc996981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:26.430712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac1681-9b49-44b8-967c-a22bc996981e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:26.430996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac1681-9b49-44b8-967c-a22bc996981e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:26.431272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac1681-9b49-44b8-967c-a22bc996981e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.439040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac1681-9b49-44b8-967c-a22bc996981e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:26.440588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac1681-9b49-44b8-967c-a22bc996981e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2aac1681-9b49-44b8-967c-a22bc996981e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:26.452748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:26.453949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:26.455002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 362/728] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:26.455601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b7c08-856a-4a2c-841c-e45701787ac2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:26.456040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b7c08-856a-4a2c-841c-e45701787ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:26.456040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b7c08-856a-4a2c-841c-e45701787ac2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.457613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 363/729] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:26.459137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b7c08-856a-4a2c-841c-e45701787ac2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:26.488546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:26.514379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 367/730] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:56:26 2022] POST /identity/v3/services => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:56:26.516083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ec17-2807-4c5e-8e8b-dc123b16da3c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:26.516647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ec17-2807-4c5e-8e8b-dc123b16da3c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:26.517450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ec17-2807-4c5e-8e8b-dc123b16da3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.543890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a536ec17-2807-4c5e-8e8b-dc123b16da3c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:26.546265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a536ec17-2807-4c5e-8e8b-dc123b16da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:26.597291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:26.649973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 368/731] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:26.665237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b7c08-856a-4a2c-841c-e45701787ac2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:26.712388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:26.712932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:26.713212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:26.713484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.714306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:26.714615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:26.716290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:26.734837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 364/732] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Sep 4 15:56:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:26.739993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536ec17-2807-4c5e-8e8b-dc123b16da3c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:26.851925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536ec17-2807-4c5e-8e8b-dc123b16da3c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:26.889321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5a441-15b2-4a76-84fa-4f8a72519bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:26.890306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a441-15b2-4a76-84fa-4f8a72519bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:26.890773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a441-15b2-4a76-84fa-4f8a72519bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:26.891263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a441-15b2-4a76-84fa-4f8a72519bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:26.919732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:26.959076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:26.972673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5a441-15b2-4a76-84fa-4f8a72519bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:26.974474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5a441-15b2-4a76-84fa-4f8a72519bc7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:27.118466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:27.121791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 369/733] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:26 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 15:56:27.123133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4958-12aa-4150-840e-865fa8acb0ca None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.123424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4958-12aa-4150-840e-865fa8acb0ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.123696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4958-12aa-4150-840e-865fa8acb0ca None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.125224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 370/734] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:27.126912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4958-12aa-4150-840e-865fa8acb0ca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.141877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b436c3-660a-47bb-85e2-65f121c629c4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:27.182151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75643db0-342c-46f9-8811-30a31c1a5a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:27.183159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75643db0-342c-46f9-8811-30a31c1a5a38 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.183621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75643db0-342c-46f9-8811-30a31c1a5a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.184277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75643db0-342c-46f9-8811-30a31c1a5a38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.187225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:27.197069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75643db0-342c-46f9-8811-30a31c1a5a38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-75643db0-342c-46f9-8811-30a31c1a5a38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:27.202339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:27.204873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:27.205738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:27.206957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 365/735] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:27.208488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3987ea5-4aa6-4b77-ac7e-6cee9209793a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.208973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3987ea5-4aa6-4b77-ac7e-6cee9209793a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.209443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3987ea5-4aa6-4b77-ac7e-6cee9209793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.230945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c3987ea5-4aa6-4b77-ac7e-6cee9209793a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:27.232727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c3987ea5-4aa6-4b77-ac7e-6cee9209793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:27.238670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4958-12aa-4150-840e-865fa8acb0ca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:27.275061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:27.275588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.275864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.276130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.287319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:27.289700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:27.305535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:27.306622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:27.307525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:27.307525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:27.307525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:27.307525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 371/736] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:27.308436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.318291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 366/737] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 15:56:27.319438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3987ea5-4aa6-4b77-ac7e-6cee9209793a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.411097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:27.435359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3987ea5-4aa6-4b77-ac7e-6cee9209793a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:27.472431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:27.473288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.474723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.475151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.475997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:27.476695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:27.479371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:27.502929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 367/738] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:27.508988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.548126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a232b-84ce-4226-accd-ba8264a3a511 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:27.560641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:27.616305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:27.617089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.617371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.617681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.618353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:27.618655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:27.620039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:27.629076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 372/739] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:27.629668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.669148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad450a0-10a4-4bcd-816c-97b182382fbe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:27.701454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:27.707879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:27.709004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.709449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.719191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.719191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:27.719731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:27.735062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01eecd-e304-4de2-8439-18d73a12aa15 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:27.736063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:27.736939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:27.737878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:27.738661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 368/740] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:27.739723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.781245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:27.782236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.782938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.783640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.784514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:27.786218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:27.859783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe0f7b-552c-4c5d-b05d-1db062205288 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:27.859995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 373/741] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:56:27 2022] POST /identity/v3/services => generated 240 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:27.863466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.898753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:27.919189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30610b2b-1f98-4fcb-b127-2d864cc5743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:27.919839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30610b2b-1f98-4fcb-b127-2d864cc5743d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.920121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30610b2b-1f98-4fcb-b127-2d864cc5743d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.920394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30610b2b-1f98-4fcb-b127-2d864cc5743d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.933686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30610b2b-1f98-4fcb-b127-2d864cc5743d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:27.935368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30610b2b-1f98-4fcb-b127-2d864cc5743d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-30610b2b-1f98-4fcb-b127-2d864cc5743d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:27.952710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:27.953779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:27.954701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:27.954701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:27.954701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:27.954701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 369/742] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:27.955812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860fb6-d13e-4e3d-aa52-2a3dd4fcc6c8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:27.956100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860fb6-d13e-4e3d-aa52-2a3dd4fcc6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:27.956379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860fb6-d13e-4e3d-aa52-2a3dd4fcc6c8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:27.958111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 370/743] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:27.963813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2860fb6-d13e-4e3d-aa52-2a3dd4fcc6c8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:27.991601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913532f9-d206-445f-823e-36e64e7c229d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:28.011153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:28.034229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a7723-2443-4546-9e93-bb37cc84b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:28.035338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a7723-2443-4546-9e93-bb37cc84b62f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:28.035857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a7723-2443-4546-9e93-bb37cc84b62f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:28.036478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a7723-2443-4546-9e93-bb37cc84b62f None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:28.038091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a7723-2443-4546-9e93-bb37cc84b62f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:28.038777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a7723-2443-4546-9e93-bb37cc84b62f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:28.040477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a7723-2443-4546-9e93-bb37cc84b62f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:28.056574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 374/744] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:28.057929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b6e3-953c-4071-b212-daa7c69399b7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:28.058214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b6e3-953c-4071-b212-daa7c69399b7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:28.058489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b6e3-953c-4071-b212-daa7c69399b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:28.078253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0d71b6e3-953c-4071-b212-daa7c69399b7 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:28.079444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0d71b6e3-953c-4071-b212-daa7c69399b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:28.095873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2860fb6-d13e-4e3d-aa52-2a3dd4fcc6c8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:28.139425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:28.140094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:28.140372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:28.140692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:28.141391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:28.141730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:28.143143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:28.157191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 371/745] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:28.158105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:28.170097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 375/746] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:28.170964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b6e3-953c-4071-b212-daa7c69399b7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:28.192485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:28.286169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b6e3-953c-4071-b212-daa7c69399b7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:28.329527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:28.333075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:28.333361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:28.333664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:28.342358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:28.343826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:28.357369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:28.358373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:28.359319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 376/747] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:56:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:28.367763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:28.413689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:28.417240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58119a0a-9b20-4d77-a6eb-da8a8da99c52 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:28.455873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:28.456512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:28.456795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:28.457071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:28.457831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=91bc1f3a0e9046bab7aa2c799ad46c3b) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:28.458145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:28.459713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:28.466515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 372/748] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:28 2022] GET /identity/v3/role_assignments?user.id=91bc1f3a0e9046bab7aa2c799ad46c3b&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:28.512970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:28.516136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:28.573550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ecd9-4fbe-4d04-926c-e85fe81d137d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:28.645367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:28.646174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:28.646379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:28.646659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:28.648446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:28.648988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:28.652315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:28.675549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 377/749] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Sep 4 15:56:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:28.685600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:28.697275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8147-c2fc-4a79-83da-cda8f204d99a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:28.771691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:28.772873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:28.773195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:28.773522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:28.784962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:28.791113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:28.792095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:28.792928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 373/750] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:28.798016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:28.852847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:28.941514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05b1c-c9b9-4b23-9f7d-e00ad2403115 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:28.985051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0f222-b534-4dc8-b4a8-aa96bad59e5e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:29.008723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc56fc6-79da-4cd2-8162-6c2006eb4228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.009672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc56fc6-79da-4cd2-8162-6c2006eb4228 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc56fc6-79da-4cd2-8162-6c2006eb4228 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.010244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc56fc6-79da-4cd2-8162-6c2006eb4228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.015417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:29.031148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30dd23-680f-4e5c-b444-ae54158b2b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.032626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30dd23-680f-4e5c-b444-ae54158b2b1a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.033167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30dd23-680f-4e5c-b444-ae54158b2b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.033619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30dd23-680f-4e5c-b444-ae54158b2b1a None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.034627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30dd23-680f-4e5c-b444-ae54158b2b1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:29.035384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30dd23-680f-4e5c-b444-ae54158b2b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.036912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30dd23-680f-4e5c-b444-ae54158b2b1a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.050429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 374/751] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:29.052717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969a065-ea59-45f1-a87b-69f56feab6c2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.053162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969a065-ea59-45f1-a87b-69f56feab6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.053759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969a065-ea59-45f1-a87b-69f56feab6c2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.056701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 375/752] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:29.057682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969a065-ea59-45f1-a87b-69f56feab6c2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.104248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc56fc6-79da-4cd2-8162-6c2006eb4228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.105830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc56fc6-79da-4cd2-8162-6c2006eb4228 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.178948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:29.196182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969a065-ea59-45f1-a87b-69f56feab6c2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:29.233754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.234562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.234958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.235377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.237481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 378/753] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:56:29.239402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d16e6-7b4c-4585-a833-086e796660a5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.239857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d16e6-7b4c-4585-a833-086e796660a5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.240303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d16e6-7b4c-4585-a833-086e796660a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.247596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.249774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.262947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9a7d16e6-7b4c-4585-a833-086e796660a5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:29.264325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9a7d16e6-7b4c-4585-a833-086e796660a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:29.267829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:29.269026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:29.270156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:29.270156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:29.270156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:29.271011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 376/754] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:29.271887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.305099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:29.360881 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 379/755] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:29.361771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d16e6-7b4c-4585-a833-086e796660a5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.394706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b8479-2714-4fcf-9ff4-dc1508f28272 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:29.443125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.443749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.449591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.449591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.459567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:29.470904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.472374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:29.501296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:29.502352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:29.503250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:29.503250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:29.503250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:29.505910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 377/756] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:29.510700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.525857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d16e6-7b4c-4585-a833-086e796660a5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:29.576467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.577154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.577438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.577741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.578396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:29.578698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.580069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.605801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 380/757] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:29.611708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.666395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:29.711392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e45e3-234d-4ca9-97db-4397a9b67f60 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:29.749536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.750345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.750731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.751369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.752151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:29.752588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.754112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.756883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fd9ae-ad31-4c3d-a42e-bcfc6dcebe06 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:29.766843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 378/758] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:29.767571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.826374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.827129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.827482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.827832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.838907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.839330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:29.840998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:29.855699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:29.856848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:29.858042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:29.861639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 381/759] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:29.862642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.905627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c8875-f9df-4086-a114-382ea375a974 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:29.948861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:29.949508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:29.949816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:29.950094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:29.950691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:29.952451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:29.960066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:29.963552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:29.964723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:29.965825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:29.969716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 379/760] 10.222.0.39 () {56 vars in 1129 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:29.971781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:29.986265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96928a37-9d95-4670-b89a-68358d767864 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.028887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.029845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.030135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.030412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.037503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.039780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:30.053319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:30.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:30.055411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:30.059210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 382/761] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:30.059960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.085908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9913c4-15c9-443d-990c-08a641e3d193 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.099716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:30.121750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56686b0-db44-407e-a9a3-920f2267e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.122299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56686b0-db44-407e-a9a3-920f2267e75b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.122577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56686b0-db44-407e-a9a3-920f2267e75b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.122851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56686b0-db44-407e-a9a3-920f2267e75b None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.123581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56686b0-db44-407e-a9a3-920f2267e75b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:30.123883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56686b0-db44-407e-a9a3-920f2267e75b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.125302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56686b0-db44-407e-a9a3-920f2267e75b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.143265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 380/762] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:56:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:30.144584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01ece-4710-4e80-9c17-b396e9ea946c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.144868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01ece-4710-4e80-9c17-b396e9ea946c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.145141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01ece-4710-4e80-9c17-b396e9ea946c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.149030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 381/763] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:30.149030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01ece-4710-4e80-9c17-b396e9ea946c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.176853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54a16-8d07-4954-ad6b-b0aa4c56543e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.213205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.213844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.214126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.214400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.215035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:30.215334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.216701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.221601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 383/764] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:30.222176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.234701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:30.264725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01ece-4710-4e80-9c17-b396e9ea946c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.300304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbad3ac-edaa-4ab6-87a9-347a9a942dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.300911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbad3ac-edaa-4ab6-87a9-347a9a942dee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.301190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbad3ac-edaa-4ab6-87a9-347a9a942dee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.301462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbad3ac-edaa-4ab6-87a9-347a9a942dee None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.302210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbad3ac-edaa-4ab6-87a9-347a9a942dee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:30.302555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbad3ac-edaa-4ab6-87a9-347a9a942dee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.304088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbad3ac-edaa-4ab6-87a9-347a9a942dee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.320428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 382/765] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:30.320428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b1d32-1910-48e1-87ed-fa6162bc7273 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.320428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b1d32-1910-48e1-87ed-fa6162bc7273 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.320428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b1d32-1910-48e1-87ed-fa6162bc7273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.336659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9f0b1d32-1910-48e1-87ed-fa6162bc7273 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:30.337987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9f0b1d32-1910-48e1-87ed-fa6162bc7273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:30.342672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba887e33-0bde-4c3b-9ca3-9228a0118646 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.362670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:30.380534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.381103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.381388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.381747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.393181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.394920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.409230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 383/766] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:30.410224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b1d32-1910-48e1-87ed-fa6162bc7273 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:30.411267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:30.412368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:30.413840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:30.413840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:30.413840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:30.415203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 384/767] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:30.416215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.501201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:30.532591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b1d32-1910-48e1-87ed-fa6162bc7273 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.557607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78326241-7e91-46b4-92c4-66ea31d15451 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.578146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.579067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.579551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.580484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.581556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:30.582055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.584286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.593266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 384/768] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:30.594475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.594768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.595465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.595803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.596148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.596951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=4fb881e220f041f1a7c99960be9aeb2c) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:30.597330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.598887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.613590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 385/769] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/role_assignments?user.id=4fb881e220f041f1a7c99960be9aeb2c&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:30.613590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.634959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:30.741327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f3ffd-2a47-4e97-afed-815a90f28a8c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.748905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03273e5-6f86-486a-8667-e63626a3e007 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.776863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:30.783935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.784544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.784823 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.785141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.785804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:30.791993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.792110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50a66e-5537-47e1-9d98-6307857a1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.792110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50a66e-5537-47e1-9d98-6307857a1af6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.792110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50a66e-5537-47e1-9d98-6307857a1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.792337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.792412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50a66e-5537-47e1-9d98-6307857a1af6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.792412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50a66e-5537-47e1-9d98-6307857a1af6 None admin] RBAC: Adding query filter params (interface=public, service_id=fa3b8f0e5c914b5f8206fe7efdf575ad, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:30.792412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50a66e-5537-47e1-9d98-6307857a1af6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.792412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50a66e-5537-47e1-9d98-6307857a1af6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.803414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 386/770] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/endpoints?service_id=fa3b8f0e5c914b5f8206fe7efdf575ad&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:30.803414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9e330-8333-45d7-8aa4-b1c74e609476 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.803414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9e330-8333-45d7-8aa4-b1c74e609476 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.803414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9e330-8333-45d7-8aa4-b1c74e609476 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.804924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 385/771] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:30.805717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.806748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 387/772] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:30.807437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9e330-8333-45d7-8aa4-b1c74e609476 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.890938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:30.907761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54424a-5fac-43e4-a3bd-e7ccbd13e254 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.921992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9e330-8333-45d7-8aa4-b1c74e609476 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:30.944480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.945211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.945501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.946067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] PATH_INFO: `/v3/services/network` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.946394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.947756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:30.956680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:30.957695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:30.962733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:30.963519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a53e7-6a5b-4945-aec8-4826ea54cc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:30.963519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a53e7-6a5b-4945-aec8-4826ea54cc55 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.963519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a53e7-6a5b-4945-aec8-4826ea54cc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.963519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a53e7-6a5b-4945-aec8-4826ea54cc55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:30.963785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 386/773] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:30.963785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:30.970594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a53e7-6a5b-4945-aec8-4826ea54cc55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:30.972054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a53e7-6a5b-4945-aec8-4826ea54cc55 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c06a53e7-6a5b-4945-aec8-4826ea54cc55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:30.984893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:30.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 388/774] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ede93b-b454-41e7-8787-51a3440e9e67 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:30.992232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ede93b-b454-41e7-8787-51a3440e9e67 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:30.992843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ede93b-b454-41e7-8787-51a3440e9e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.008153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-63ede93b-b454-41e7-8787-51a3440e9e67 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:31.009349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-63ede93b-b454-41e7-8787-51a3440e9e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:31.064988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:31.078298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ee10f-5fc6-446d-81c6-d6547cee7c09 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.086631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 389/775] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:31.087313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ede93b-b454-41e7-8787-51a3440e9e67 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.128221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.129599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.130027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.130471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.132338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:31.134284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:31.142326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 387/776] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:56:30 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:31.143097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.169998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:31.267604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97832302-08e0-46fd-a3ea-b5c6e5b8446d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.280707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ede93b-b454-41e7-8787-51a3440e9e67 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.310941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.312193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.312622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.313065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.314199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:31.314655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:31.316970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:31.341619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 388/777] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:31.341619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.366949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.367586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.367874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.368152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.368796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] RBAC: Adding query filter params (name=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:31.369101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:31.370584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:31.391398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 390/778] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:31.400557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.437956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:31.455569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:31.469980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f52a36-d497-4231-ad5e-e936d04b55a6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.507689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.508804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.509191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.509602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/46fd540df66143b2b768cc8d9f4beed7/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.512314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=46fd540df66143b2b768cc8d9f4beed7, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:31.513893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:31.524797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a88df-ecfa-4de0-9510-0dfe77da8b8c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.562631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.563238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.563561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.563830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.564454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:31.564751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:31.566167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:31.573654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 391/779] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:31.574788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.590862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 389/780] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:31 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/46fd540df66143b2b768cc8d9f4beed7/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 15:56:31.598571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.779250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce679a5-5781-448b-afe8-cb3c45020229 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.793065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdbe64-28f8-44c3-845c-117bb029aed0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.804322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:31.816766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.822507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.822507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.822507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.822507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] RBAC: Adding query filter params (interface=public, service_id=387f8a1745424dc89acd1cbc2fbacc46, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:31.822507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:31.822507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:31.829211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 390/781] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:56:31 2022] GET /identity/v3/endpoints?service_id=387f8a1745424dc89acd1cbc2fbacc46&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:31.829833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:31.837821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.841220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.842165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.842650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.843123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.855683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:31.860998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:31.861950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:31.862792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:31.865332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 392/782] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:31.872427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:31.958117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a9269-3212-4786-a2ac-b52c96119aa6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.981996 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71066ea5-7ee1-4cc2-800e-f945d4be0c20 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:31.993508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:31.994300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:31.994483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:31.994707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:31.995312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:31.996665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:32.002603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 391/783] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:56:31 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:32.040038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:32.040911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:32.041412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:32.041870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:32.042693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:32.043146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:32.044668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:32.061602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 393/784] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:32.067619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:32.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:32.215684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c19b-8d09-425a-bdac-80f8c9e12764 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:32.255887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:32.257092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:32.257374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:32.257743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:32.272998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:32.274601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:32.292900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:32.293912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:32.294815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:32.294815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:32.294815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:32.294815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 392/785] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:32.301147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:32.364334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:32.461236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de58d-0705-418f-ba86-64f00300929f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:32.503429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:32.504042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:32.504322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:32.504593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:32.505231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:32.505529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:32.507153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:32.521981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 394/786] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:32.528258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:32.645324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbf0ce-70e7-47b0-a917-d99613b02560 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:32.663682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:32.696080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:32.696722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:32.697000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:32.697268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:32.707745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:32.709178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:32.720881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:32.721868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:32.722909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 393/787] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:32.730185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:32.842519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:32.852759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f50fc-aa01-491a-9a1e-a2d744d298b3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:32.896608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:32.897215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:32.897498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:32.898045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:32.898578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:32.898886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:32.900329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:32.925250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 395/788] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Sep 4 15:56:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:32.930627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:33.070766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc4a0f-be4c-45e5-865f-c56b63fea09a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:33.114401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b0d67-e42b-4c45-9905-30ce9b35236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:33.115036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b0d67-e42b-4c45-9905-30ce9b35236a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.115363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b0d67-e42b-4c45-9905-30ce9b35236a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.115634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b0d67-e42b-4c45-9905-30ce9b35236a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.132964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b0d67-e42b-4c45-9905-30ce9b35236a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:33.134491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b0d67-e42b-4c45-9905-30ce9b35236a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1b5b0d67-e42b-4c45-9905-30ce9b35236a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:33.150915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:33.151863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:33.152774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:33.152774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:33.152774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:33.152774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 394/789] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:33.157960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:33.213299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22601-f987-4483-a286-2065ce1b4ba7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.213633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22601-f987-4483-a286-2065ce1b4ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.213894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22601-f987-4483-a286-2065ce1b4ba7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.215468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 395/790] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:33.220300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19adb5-6f87-4390-aaba-918dda937151 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.220681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19adb5-6f87-4390-aaba-918dda937151 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.221051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19adb5-6f87-4390-aaba-918dda937151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.241068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3c19adb5-6f87-4390-aaba-918dda937151 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:33.242439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3c19adb5-6f87-4390-aaba-918dda937151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:33.273961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:33.320436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9b0c-bedb-4ef7-8c53-c4e037ec2609 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:33.371620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 396/791] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:33.381812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32ac0d-b9b1-4b4c-a7aa-a4e5527b428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:33.382429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32ac0d-b9b1-4b4c-a7aa-a4e5527b428b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.382701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32ac0d-b9b1-4b4c-a7aa-a4e5527b428b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.382970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32ac0d-b9b1-4b4c-a7aa-a4e5527b428b None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.383606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32ac0d-b9b1-4b4c-a7aa-a4e5527b428b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:33.383903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32ac0d-b9b1-4b4c-a7aa-a4e5527b428b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:33.385260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32ac0d-b9b1-4b4c-a7aa-a4e5527b428b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:33.398151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 396/792] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:33.406741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19adb5-6f87-4390-aaba-918dda937151 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:33.453465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c717-a6ba-46c0-a383-9e8cca4db7e1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.454136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c717-a6ba-46c0-a383-9e8cca4db7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.454621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c717-a6ba-46c0-a383-9e8cca4db7e1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.456421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 397/793] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:33.462611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259da35-64d5-4519-a744-6729d6b61de9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.463170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259da35-64d5-4519-a744-6729d6b61de9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.463654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259da35-64d5-4519-a744-6729d6b61de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.483759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8259da35-64d5-4519-a744-6729d6b61de9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:33.485402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8259da35-64d5-4519-a744-6729d6b61de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:33.498050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:33.585819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19adb5-6f87-4390-aaba-918dda937151 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:33.607344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 398/794] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:33.633713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:33.634414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.634414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.635247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.642169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:33.643614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:33.655790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:33.657170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:33.658227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 397/795] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:33.662931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259da35-64d5-4519-a744-6729d6b61de9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:33.754706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:33.785097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:33.801885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259da35-64d5-4519-a744-6729d6b61de9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:33.872479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:33.873148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:33.873389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:33.873671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:33.874376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:33.874684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:33.876393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:33.888126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:33.894723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 399/796] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:33.900732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:33.993212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354920e-4045-4e3f-b1d2-073c9ab8ab9b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.014349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cbf9e-3970-4f1a-9b5f-e1029719e3af None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.058319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.059588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.060514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.061267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.065140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e6507-c3c1-4f6e-8f42-8751298ae410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.065763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e6507-c3c1-4f6e-8f42-8751298ae410 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.066118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e6507-c3c1-4f6e-8f42-8751298ae410 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.066470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e6507-c3c1-4f6e-8f42-8751298ae410 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/91bc1f3a0e9046bab7aa2c799ad46c3b/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.068564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e6507-c3c1-4f6e-8f42-8751298ae410 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=91bc1f3a0e9046bab7aa2c799ad46c3b, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:34.070422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e6507-c3c1-4f6e-8f42-8751298ae410 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:34.076399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:34.082384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:34.086806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:34.087676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 398/797] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:34.089522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.139230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 400/798] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:33 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/91bc1f3a0e9046bab7aa2c799ad46c3b/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:56:34.140773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ef258-88f0-4385-bc27-6cfe9eb7918e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.141232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ef258-88f0-4385-bc27-6cfe9eb7918e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.141702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ef258-88f0-4385-bc27-6cfe9eb7918e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.143587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 401/799] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:34.144610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ef258-88f0-4385-bc27-6cfe9eb7918e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.179118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:34.207749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36f680-33f9-4cc6-8cf8-759da0399adb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.228101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:34.252025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcd9cc-0896-40cb-9efe-66237135e3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.253011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcd9cc-0896-40cb-9efe-66237135e3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.253470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcd9cc-0896-40cb-9efe-66237135e3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.253922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcd9cc-0896-40cb-9efe-66237135e3d0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.254669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcd9cc-0896-40cb-9efe-66237135e3d0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:34.256263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcd9cc-0896-40cb-9efe-66237135e3d0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-dddcd9cc-0896-40cb-9efe-66237135e3d0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:34.267380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:34.268609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 15:56:34.269550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:34.272905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 399/800] 10.222.0.39 () {56 vars in 1129 bytes} [Sun Sep 4 15:56:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:34.274690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719cbf-6a29-4ba8-ba6b-f596f72fbd56 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.275161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719cbf-6a29-4ba8-ba6b-f596f72fbd56 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.276377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719cbf-6a29-4ba8-ba6b-f596f72fbd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.295509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ef258-88f0-4385-bc27-6cfe9eb7918e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.296496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0a719cbf-6a29-4ba8-ba6b-f596f72fbd56 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:34.298222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0a719cbf-6a29-4ba8-ba6b-f596f72fbd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:34.337821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.338611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.339201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.339630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.340292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:34.340775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:34.342542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:34.354307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 402/801] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:34.355163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.404325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 400/802] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:34.405484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a719cbf-6a29-4ba8-ba6b-f596f72fbd56 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.481332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e373f-27a9-48f1-ad21-1a6c6065be26 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.510539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:34.524763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:34.531144 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c3660-280f-421c-9a56-4061d095984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.531888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3660-280f-421c-9a56-4061d095984b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.532174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3660-280f-421c-9a56-4061d095984b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.532458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3660-280f-421c-9a56-4061d095984b None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.533148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c3660-280f-421c-9a56-4061d095984b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:34.533464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c3660-280f-421c-9a56-4061d095984b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:34.535382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c3660-280f-421c-9a56-4061d095984b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:34.542858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 403/803] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Sep 4 15:56:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:34.545332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b76-a03a-4f19-8922-530c69d352fb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.545644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b76-a03a-4f19-8922-530c69d352fb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.545930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b76-a03a-4f19-8922-530c69d352fb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.547778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 404/804] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:34.557514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547b76-a03a-4f19-8922-530c69d352fb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.563899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a719cbf-6a29-4ba8-ba6b-f596f72fbd56 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.610795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbce7b-60d5-4dea-9fe8-e7b60e48ecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.611500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbce7b-60d5-4dea-9fe8-e7b60e48ecd9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.611783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbce7b-60d5-4dea-9fe8-e7b60e48ecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.612054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbce7b-60d5-4dea-9fe8-e7b60e48ecd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.624467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbce7b-60d5-4dea-9fe8-e7b60e48ecd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:34.626353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbce7b-60d5-4dea-9fe8-e7b60e48ecd9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-dddbce7b-60d5-4dea-9fe8-e7b60e48ecd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:34.643977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:34.644981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:34.645898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:34.645898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:34.645898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:34.645898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 401/805] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:34.648114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e368-01c5-4e58-aba1-8fb58f6b0820 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.648114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e368-01c5-4e58-aba1-8fb58f6b0820 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.648114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e368-01c5-4e58-aba1-8fb58f6b0820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.668077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-66b7e368-01c5-4e58-aba1-8fb58f6b0820 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:34.669468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-66b7e368-01c5-4e58-aba1-8fb58f6b0820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:34.751888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 402/806] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:34.752734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e368-01c5-4e58-aba1-8fb58f6b0820 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.781988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547b76-a03a-4f19-8922-530c69d352fb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.825772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:34.829164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.830297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.830767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.831381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.832566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:34.833297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:34.836052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:34.843293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 405/807] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Sep 4 15:56:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:34.844636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.878869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e368-01c5-4e58-aba1-8fb58f6b0820 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:34.913994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:34.914513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:34.915307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:34.915307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:34.915576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:34.916211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:34.916511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:34.917905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:34.930545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 403/808] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:34.931353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:34.978039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926a354-7c6a-40fb-836c-cbb17fe1cedf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.023999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.025226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.025769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.026442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.039907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:35.044995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:35.046121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:35.046967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:35.047608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0965ff7-413a-42c3-aaa3-a9ecb526e074 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.047684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 406/809] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:35.047967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.058691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:35.081999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.082603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.082880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.083147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.083799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.085427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.153505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd419c56-b6e1-41b8-a0c2-722f65b4a6f6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.163104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:35.179886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 404/810] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:56:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:35.180975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.208381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.209166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.209549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.210113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.219895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.222320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:35.235925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:35.236989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:35.237973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:35.242556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 407/811] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:35.243575 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.342993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9319a-3e5b-4ca3-a246-883d1b8916d7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.386583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314918c-f2c5-4780-9510-f0c1f253468a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.387068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.388050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.388318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.388597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] PATH_INFO: `/v3/services/network` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.389311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.390913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:35.401626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:35.402696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:35.403534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 405/812] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:35.404344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.444982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.446195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.447047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:35.447336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.448384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.449448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:35.450245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.451839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.463265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 408/813] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:35.467268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.499820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:35.543584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc42f-ca7a-494c-9a70-b554aed9643f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.592017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.592708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.592997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.593277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.594079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:35.594451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.595513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038932c-3fcc-4ef1-9c69-3fa9ec222089 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.595946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.626336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 406/814] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:35.627259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.644970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.645609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.646155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.646439 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.647990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] RBAC: Adding query filter params (name=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:35.648302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.649755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.656808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 409/815] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:35.657713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.735669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:35.748747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2b19c-0d90-4570-b41f-5e0af0261d31 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.766329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:35.777324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502778-6188-4d64-bb65-547355242f5b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.786882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.788405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.788405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.788690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.799679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.801124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.815880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.816790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.817220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.817702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:35.818761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:35.820753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:35.821982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:35.821982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:35.821982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:35.825426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 407/816] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:35.826320 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.833106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.835446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:35.853548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:35.854769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:35.855660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:35.855660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:35.855660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:35.857416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 410/817] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:35.858581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.940382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78acfaa-cddf-4318-a5ce-75d195bd1997 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.966500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda07850-2953-4fc5-bb03-252480e7cfc3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:35.978873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:35.979358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:35.979647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:35.979927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:35.980554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] RBAC: Adding query filter params (type=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:35.980881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:35.982270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:35.992796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 408/818] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:35.992796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:35.992796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.012082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.013143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.013737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.014217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.015062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:36.015585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.017167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.030582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 411/819] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:36.031636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:36.037225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.116300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d30127-1100-4f4c-9619-6a42796e287a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.143871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6a7a3-139f-4277-ad56-584c026a98b2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.163448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.164398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.164957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.166029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.167191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:36.167713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.170220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.180769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.181355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.181667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.181947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.182626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.183997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.187598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 409/820] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:36.188764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:36.261608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 412/821] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:56:36 2022] POST /identity/v3/endpoints => generated 337 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:36.261608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:36.264580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.305279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12abf45-1358-452f-a0dd-233a3ad35d94 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.340415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.357727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447238f9-06e2-4fec-9382-b79fcdd31aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.358385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447238f9-06e2-4fec-9382-b79fcdd31aea None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.358661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447238f9-06e2-4fec-9382-b79fcdd31aea None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.358992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447238f9-06e2-4fec-9382-b79fcdd31aea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.367896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447238f9-06e2-4fec-9382-b79fcdd31aea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.369425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447238f9-06e2-4fec-9382-b79fcdd31aea None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-447238f9-06e2-4fec-9382-b79fcdd31aea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:36.383178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:36.384232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:36.385178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 410/822] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:56:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:36.385794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e4c5c-eed1-4e85-b0fc-511af7e75370 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.385875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e4c5c-eed1-4e85-b0fc-511af7e75370 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.386395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e4c5c-eed1-4e85-b0fc-511af7e75370 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.392600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 411/823] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:36.393459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e4c5c-eed1-4e85-b0fc-511af7e75370 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:36.397985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c49db2-9be6-4543-917f-d0825e6814aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.440003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0d9a1-0718-47df-9f87-7eb65837f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.440956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0d9a1-0718-47df-9f87-7eb65837f1aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.441279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0d9a1-0718-47df-9f87-7eb65837f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.441578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0d9a1-0718-47df-9f87-7eb65837f1aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.449962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0d9a1-0718-47df-9f87-7eb65837f1aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.451409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0d9a1-0718-47df-9f87-7eb65837f1aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-06c0d9a1-0718-47df-9f87-7eb65837f1aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:36.465392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:36.466362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:36.467635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 413/824] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:36.468306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae52bb-94c5-4fab-902f-97ddc7b90c64 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.468912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae52bb-94c5-4fab-902f-97ddc7b90c64 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.469250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae52bb-94c5-4fab-902f-97ddc7b90c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.489109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8bae52bb-94c5-4fab-902f-97ddc7b90c64 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:36.490440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8bae52bb-94c5-4fab-902f-97ddc7b90c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:36.530562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e4c5c-eed1-4e85-b0fc-511af7e75370 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.553763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.569914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.578521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.578521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.578521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.578521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:36.578521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.578521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.579077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 414/825] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:36.579077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae52bb-94c5-4fab-902f-97ddc7b90c64 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:36.595613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 412/826] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:56:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:36.601903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:36.634684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.702627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae52bb-94c5-4fab-902f-97ddc7b90c64 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.743345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e9244-220d-4b73-8157-a92b39db637f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.748696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.749388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.749768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.750112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.750883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:36.751520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.753464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.770627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 415/827] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:36.777488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:36.788665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.789276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.789552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.789863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.836666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.859348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.860790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.890794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1842b-55ac-402a-b08c-05f3bf83fb78 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:36.901647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:36.929183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:36.929898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:36.930184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:36.930463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/4fb881e220f041f1a7c99960be9aeb2c/roles/fa94d66c831d468f83e224236c343293` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:36.945779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=4fb881e220f041f1a7c99960be9aeb2c, role_id=fa94d66c831d468f83e224236c343293)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:36.947232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:36.978414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 413/828] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:56:36.979272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.008753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 416/829] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:36 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/4fb881e220f041f1a7c99960be9aeb2c/roles/fa94d66c831d468f83e224236c343293 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:56:37.009647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.083897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:37.109402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bd127-cc2f-467b-ad33-d0568762677c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.152931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.153700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.154083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.154473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.156067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9754c-09f9-461f-8861-34a6b354ab42 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.174279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:37.180197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:37.181140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:37.182617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 414/830] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:37.187112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.215572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e699d-47eb-4cc5-bed5-35e699d9cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.216217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e699d-47eb-4cc5-bed5-35e699d9cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.216496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e699d-47eb-4cc5-bed5-35e699d9cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.216765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e699d-47eb-4cc5-bed5-35e699d9cdf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.234845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e699d-47eb-4cc5-bed5-35e699d9cdf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:37.236633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e699d-47eb-4cc5-bed5-35e699d9cdf8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9c6e699d-47eb-4cc5-bed5-35e699d9cdf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:37.252435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:37.254055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:37.254949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:37.254949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:37.254949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:37.254949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 417/831] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:37.255212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20103d5d-25a6-49bb-b5a0-7083c5448328 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.255429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20103d5d-25a6-49bb-b5a0-7083c5448328 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.255704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20103d5d-25a6-49bb-b5a0-7083c5448328 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.257209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 418/832] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:37.262039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20103d5d-25a6-49bb-b5a0-7083c5448328 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.276662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:37.317751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c38c3-4950-4932-a27a-ef373cab115f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.360388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db52d-e817-4d6b-a98a-2b3e573a1057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.361096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db52d-e817-4d6b-a98a-2b3e573a1057 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.361479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db52d-e817-4d6b-a98a-2b3e573a1057 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.361850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db52d-e817-4d6b-a98a-2b3e573a1057 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.362528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db52d-e817-4d6b-a98a-2b3e573a1057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:37.362836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db52d-e817-4d6b-a98a-2b3e573a1057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:37.364201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db52d-e817-4d6b-a98a-2b3e573a1057 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:37.369458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:37.373313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 415/833] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:37.374825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bfd5b-9d47-4cb5-9bad-44e79b9aa908 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.375302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bfd5b-9d47-4cb5-9bad-44e79b9aa908 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.375740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bfd5b-9d47-4cb5-9bad-44e79b9aa908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.384301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20103d5d-25a6-49bb-b5a0-7083c5448328 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.400066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-045bfd5b-9d47-4cb5-9bad-44e79b9aa908 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:37.402036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-045bfd5b-9d47-4cb5-9bad-44e79b9aa908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:37.425847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.426733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.427254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.427713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.428574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:37.429172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:37.430825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:37.448539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 419/834] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:37.449723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.473739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 416/835] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:37.474596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bfd5b-9d47-4cb5-9bad-44e79b9aa908 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.530708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:37.563901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737825f5-f1b7-4fc0-ace9-2766a11ead39 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.601917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.602537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.602884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.603230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.614350 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:37.615969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:37.621076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bfd5b-9d47-4cb5-9bad-44e79b9aa908 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:37.632474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:37.633453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:37.634400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:37.634400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:37.634400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:37.634400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 420/836] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:37.639224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.660100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:37.670121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.671277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.672548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.673171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.682988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:37.685510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:37.702450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:37.703380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:37.704274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:37.712200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 417/837] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:37.714623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.767608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbb138-1024-46b5-87ed-282bbd7c6f75 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.793489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:37.806379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.807326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.807652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.807929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.808582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:37.808886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:37.810297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:37.829163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 421/838] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:37.830868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.866934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624690bb-b144-4a12-8834-f608d1a1bf1c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:37.903163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:37.903763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:37.904048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:37.904322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:37.905129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:37.905440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:37.906887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:37.925591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 418/839] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:37.925591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:37.975642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:37.980428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8b019-949f-458f-8f70-da2383468505 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.018699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.019376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.019612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.019858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.020443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.022480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.028770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14074ad-4cdd-4c15-8d68-46d2f4f3f803 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:38.031475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:38.032460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:38.033301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 422/840] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:38.034477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:38.070037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.070702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.070987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.071255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.078989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.080815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.085449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:38.092569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:38.099254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 419/841] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:56:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:38.100772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:38.151343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff5966-f5f0-45b2-9763-33a7eb69893a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.186224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.186984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.187367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.187741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.188565 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=46fd540df66143b2b768cc8d9f4beed7) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:38.188972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.190539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.201596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 423/842] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:38 2022] GET /identity/v3/role_assignments?user.id=46fd540df66143b2b768cc8d9f4beed7&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:38.201596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:38.223166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ec4c6-9add-4b6c-93d2-6aa58ec6b34f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.235992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:38.266411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.267256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.267628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.267993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.268738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:38.269162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.270879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.276905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 420/843] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:38.282461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:38.338740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ad4c-f2fd-4713-8fcd-b3f2aaad226a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.352584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:38.382587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.383360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.385166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.385166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.385540 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:38.385877 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.387339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.406817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 424/844] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Sep 4 15:56:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:38.412485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:38.435216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865c688-231f-426d-b9e7-02d1340326f4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.476944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.477551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.477863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.478135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.478763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:38.479063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.480410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.485915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 421/845] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:38.536487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a88610-094a-4a5b-9acd-16c9fa71bf9f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.577820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.578870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.579350 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.579819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.621521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:38.657841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.659717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.771746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 425/846] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:56:38.778063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:38.827546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:38.893340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73fdfd-8380-4055-a266-c6e57186700f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:38.930496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:38.931516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:38.931996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:38.932471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:38.943158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:38.945454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:38.961520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:38.962516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:38.963405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:38.963405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:38.963405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:38.965503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 422/847] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:38.970939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:39.043542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:39.085406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60901cb5-2f1c-4c55-a632-ed0f5825d058 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:39.120882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cc8dc-d3a0-42f0-a61e-590566b1fc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:39.121505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cc8dc-d3a0-42f0-a61e-590566b1fc46 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.121816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cc8dc-d3a0-42f0-a61e-590566b1fc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.122096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cc8dc-d3a0-42f0-a61e-590566b1fc46 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.122747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cc8dc-d3a0-42f0-a61e-590566b1fc46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:39.123053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cc8dc-d3a0-42f0-a61e-590566b1fc46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:39.124486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cc8dc-d3a0-42f0-a61e-590566b1fc46 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:39.136612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 426/848] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:39.141864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:39.160571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850862-f1cf-40d6-92a7-04df2a232df1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.160871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850862-f1cf-40d6-92a7-04df2a232df1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.161145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850862-f1cf-40d6-92a7-04df2a232df1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.163348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 427/849] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:39.169041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce34d4-f410-472a-a214-d234b8f1f0ca None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.169331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce34d4-f410-472a-a214-d234b8f1f0ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.169640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce34d4-f410-472a-a214-d234b8f1f0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.188529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8cce34d4-f410-472a-a214-d234b8f1f0ca None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:39.189771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8cce34d4-f410-472a-a214-d234b8f1f0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:39.252459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:39.264763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6673a14-ed76-47a9-8341-c2ee90e5dc09 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:39.267191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 428/850] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:39.304942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:39.305922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.306352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.306950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.316264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:39.317892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:39.338296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:39.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:39.340482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 423/851] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:39.347291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce34d4-f410-472a-a214-d234b8f1f0ca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:39.419273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:39.462264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce34d4-f410-472a-a214-d234b8f1f0ca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:39.501495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:39.502141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.502418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.502689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.503381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:39.503684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:39.505166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:39.521675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 429/852] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:39.524284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:39.574053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:39.638293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a625e-f9df-48b1-a16e-ce7d1ee2a321 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:39.677701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:39.678381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.679026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.679186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.679921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=91bc1f3a0e9046bab7aa2c799ad46c3b) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:39.680276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:39.681723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:39.693590 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 424/853] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:39 2022] GET /identity/v3/role_assignments?user.id=91bc1f3a0e9046bab7aa2c799ad46c3b&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:39.708652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aa6c6-a58a-44b1-a74a-a676d6bd50f7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:39.730277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:39.749769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:39.750482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.750769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.751311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.765259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:39.771865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:39.773036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:39.773915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 430/854] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:56:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:39.779817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:39.798487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:39.900620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020c54d-17e3-4f45-9b95-c4cf6445d51f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:39.937356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:39.938003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:39.938319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:39.938607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:39.939281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:39.939586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:39.940961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:39.950232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 425/855] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:56:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:39.955872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:40.069978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3e41-ae24-40fb-af11-2326b9f7d5fc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:40.105311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:40.106366 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.106854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.107330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.119686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:40.121427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:40.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:40.137500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:40.138450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:40.138450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:40.138450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:40.140474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 431/856] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:40.146158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:40.229405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:40.254315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25403c5a-7fd9-4274-b09c-3af4c2eb837a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:40.291531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbf9a9-23f6-482d-8a20-3e878e6b0a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:40.292149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbf9a9-23f6-482d-8a20-3e878e6b0a58 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.292475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbf9a9-23f6-482d-8a20-3e878e6b0a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.292815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbf9a9-23f6-482d-8a20-3e878e6b0a58 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.293473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbf9a9-23f6-482d-8a20-3e878e6b0a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:40.293814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbf9a9-23f6-482d-8a20-3e878e6b0a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:40.295193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbf9a9-23f6-482d-8a20-3e878e6b0a58 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:40.308157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 426/857] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:40.313823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:40.335892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5327b2-1c4d-4b4b-a18b-196483ec1546 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.336111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5327b2-1c4d-4b4b-a18b-196483ec1546 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.336392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5327b2-1c4d-4b4b-a18b-196483ec1546 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.338061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 427/858] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:40.343276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b6f79-1e53-4ded-b2c9-f8d7b2661201 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.343559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b6f79-1e53-4ded-b2c9-f8d7b2661201 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.343835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b6f79-1e53-4ded-b2c9-f8d7b2661201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.361922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:40.363591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-594b6f79-1e53-4ded-b2c9-f8d7b2661201 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:40.364784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-594b6f79-1e53-4ded-b2c9-f8d7b2661201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:40.445660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 428/859] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:40.452962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9dd9-5f5d-404c-9857-cf8d515568f5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:40.488877 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:40.489876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.490386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.490889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.495687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac51d-27d1-4b0f-9457-d1c3e7670d47 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.495972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac51d-27d1-4b0f-9457-d1c3e7670d47 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.496248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac51d-27d1-4b0f-9457-d1c3e7670d47 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.497971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 429/860] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:40.500055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:40.502000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:40.503044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bcecc-9662-4048-b702-8ba5f1f3773e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.503326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bcecc-9662-4048-b702-8ba5f1f3773e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.503597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bcecc-9662-4048-b702-8ba5f1f3773e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:40.516115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:40.517102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:40.518103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:40.520618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 432/861] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:56:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:40.522993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9a5bcecc-9662-4048-b702-8ba5f1f3773e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:40.524291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9a5bcecc-9662-4048-b702-8ba5f1f3773e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:40.526758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:40.593065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:40.605703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 430/862] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:40.632782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:40.642710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bf573-af12-4cc5-ae5f-d960d0dcad02 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:40.681274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:40.681906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.682183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.682457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.683173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:40.683477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:40.684900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:40.702460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 433/863] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Sep 4 15:56:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:40.707800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bcecc-9662-4048-b702-8ba5f1f3773e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:40.727606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:40.828944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bcecc-9662-4048-b702-8ba5f1f3773e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:40.843407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef75616-df58-4169-9b28-4d357020bcd3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:40.866241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:40.866735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.867019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.867297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.882197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:40.883224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:40.883839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:40.884428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:40.885224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:40.886953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:40.897441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:40.898829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:40.899706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:40.901848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 434/864] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Sep 4 15:56:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:40.902783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:40.918875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:40.934228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:40.940728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:40.942228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.013531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b49c9-0a9b-4da7-a355-81a7f3beeb1e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:41.048640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 431/865] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:56:41.049875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:41.054996 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:41.055629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.055914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.056193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.056849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:41.058244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.130106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 435/866] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:56:40 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:41.130106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:41.153230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:41.160293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5f2a8-5777-4908-995a-858cab6bbab9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:41.207888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:41.208558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.208947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.209210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.210015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:41.210361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:41.212101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:41.213094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.218930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 432/867] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:41.225663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:41.246766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cead77-705d-4dbd-93b6-a58732805354 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:41.293100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:41.294038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.294466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.294900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.307462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:41.308922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:41.327953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:41.329146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:41.330157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:41.330157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:41.330157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:41.330157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 436/868] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:41.338192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:41.370148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa543b-afc4-484b-8cc3-237ef507932e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:41.415690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:41.416542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.416978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.417440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.418500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:41.418968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:41.421388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.427937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 433/869] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:41.464754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa6144-f8b9-409a-aa05-5d94e39b6735 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:41.493006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:41.505994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:41.506753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.507041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.507313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.511560 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:41.511560 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:41.511560 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.524812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 437/870] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:41.530123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:41.557975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:41.644400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe66081-3fc6-4f4b-83cb-5fed0ca8e702 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:41.683183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240df050-d8e6-4f37-aedc-d96ab479ef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:41.684157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240df050-d8e6-4f37-aedc-d96ab479ef6c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.684665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240df050-d8e6-4f37-aedc-d96ab479ef6c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.685172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240df050-d8e6-4f37-aedc-d96ab479ef6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.692501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240df050-d8e6-4f37-aedc-d96ab479ef6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:41.694365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240df050-d8e6-4f37-aedc-d96ab479ef6c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-240df050-d8e6-4f37-aedc-d96ab479ef6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:41.706766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:41.707699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:41.708611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:41.711815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 434/871] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:41.718305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:41.810555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28f2f-7155-42e0-8e1f-dd7460a01f1c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.811191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28f2f-7155-42e0-8e1f-dd7460a01f1c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.811704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28f2f-7155-42e0-8e1f-dd7460a01f1c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.813491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 435/872] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:41.818106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3bbe-180e-4c1e-9577-1cb1251468bd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.818605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3bbe-180e-4c1e-9577-1cb1251468bd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.819638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3bbe-180e-4c1e-9577-1cb1251468bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.823621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855202df-39c1-4fef-b94a-d67c56fa7390 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:41.840073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-412d3bbe-180e-4c1e-9577-1cb1251468bd None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:41.841844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-412d3bbe-180e-4c1e-9577-1cb1251468bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:41.859086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:41.859428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:41.859712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:41.859990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:41.860682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:41.860986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:41.862448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:41.880141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 438/873] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:41.886597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:41.925149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 436/874] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:41.938050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:42.001205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b09b1-b6f8-462e-9d09-86b408197eb8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:42.035381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:42.035985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:42.036261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:42.036534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:42.037253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=4fb881e220f041f1a7c99960be9aeb2c) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:42.038270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:42.039999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:42.046151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 439/875] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:41 2022] GET /identity/v3/role_assignments?user.id=4fb881e220f041f1a7c99960be9aeb2c&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:42.193713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3bbe-180e-4c1e-9577-1cb1251468bd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:42.206033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:42.300185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3bbe-180e-4c1e-9577-1cb1251468bd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:42.348158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:42.348796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:42.349079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:42.349346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:42.350097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:42.351527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:42.360703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:42.361925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:42.363267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 437/876] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:42.368241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:42.470166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef93c-7462-4121-9d73-2d5012adcbf0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:42.479894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:42.504796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cb2ef-8276-475f-a16d-c865a689728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:42.505415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2ef-8276-475f-a16d-c865a689728d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:42.505728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2ef-8276-475f-a16d-c865a689728d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:42.506005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2ef-8276-475f-a16d-c865a689728d None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:42.506810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb2ef-8276-475f-a16d-c865a689728d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:42.507115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb2ef-8276-475f-a16d-c865a689728d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:42.508490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb2ef-8276-475f-a16d-c865a689728d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:42.517895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 440/877] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:42.518919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:42.597759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:42.624842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107938-fb50-49a2-b997-3a3eadad3036 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:42.659702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79be01-c139-4b58-8cc2-345fa31a7d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:42.660318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79be01-c139-4b58-8cc2-345fa31a7d57 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:42.660598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79be01-c139-4b58-8cc2-345fa31a7d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:42.660874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79be01-c139-4b58-8cc2-345fa31a7d57 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:42.661505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79be01-c139-4b58-8cc2-345fa31a7d57 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:42.661839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79be01-c139-4b58-8cc2-345fa31a7d57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:42.663746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79be01-c139-4b58-8cc2-345fa31a7d57 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:42.668749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 438/878] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:43.189931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d76aa6-bd62-4b1b-bf21-689b0d04b0e0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:43.190339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d76aa6-bd62-4b1b-bf21-689b0d04b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:43.190519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d76aa6-bd62-4b1b-bf21-689b0d04b0e0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:43.192086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 441/879] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:43.197253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef99e-dd1a-4879-ac21-b80c4a50b5f3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:43.197728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef99e-dd1a-4879-ac21-b80c4a50b5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:43.197822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef99e-dd1a-4879-ac21-b80c4a50b5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:43.216843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8aeef99e-dd1a-4879-ac21-b80c4a50b5f3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:43.218060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8aeef99e-dd1a-4879-ac21-b80c4a50b5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:43.287300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 439/880] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:43.520077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5278da-5477-4b63-a79b-7d1861a879ae None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:43.520588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5278da-5477-4b63-a79b-7d1861a879ae None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:43.520588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5278da-5477-4b63-a79b-7d1861a879ae None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:43.522946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 442/881] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:43.528154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a8444-6f31-4430-a81b-82caa90f8ba3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:43.528521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a8444-6f31-4430-a81b-82caa90f8ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:43.528896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a8444-6f31-4430-a81b-82caa90f8ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:43.548294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-066a8444-6f31-4430-a81b-82caa90f8ba3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:43.549493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-066a8444-6f31-4430-a81b-82caa90f8ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:43.566970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5278da-5477-4b63-a79b-7d1861a879ae None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:43.628070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:43.635762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 440/882] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:43.686021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5278da-5477-4b63-a79b-7d1861a879ae None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:43.722450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:43.723386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:43.723847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:43.724335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:43.725204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:43.726799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:43.736293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:43.737299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:43.738166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:43.741066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 443/883] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:43.746941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a8444-6f31-4430-a81b-82caa90f8ba3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:43.855098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a8444-6f31-4430-a81b-82caa90f8ba3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:43.889950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:43.890557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:43.890882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:43.891250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:43.891881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:43.892203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:43.893555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:43.899103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 441/884] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:43.904520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:43.918114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:43.944213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:43.988344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:44.020252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e892b-a1d2-4b65-832c-4f1dcbfcb4ea None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:44.036480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcef143-641c-47ab-9423-1aaf669d792d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:44.059447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:44.060440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.061010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.061426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.062265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:44.062759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:44.064302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:44.080257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 444/885] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:44.081331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:44.086176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563ab4-bac2-4924-a15e-3c1be5429718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:44.087041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563ab4-bac2-4924-a15e-3c1be5429718 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.087517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563ab4-bac2-4924-a15e-3c1be5429718 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.088007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563ab4-bac2-4924-a15e-3c1be5429718 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.088866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45563ab4-bac2-4924-a15e-3c1be5429718 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:44.090511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45563ab4-bac2-4924-a15e-3c1be5429718 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:44.163591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 442/886] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:56:43 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:56:44.192104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:44.195790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d80e5-1bdf-4c91-945b-ea4adef486d6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:44.239071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207afca6-334e-4d6e-9aec-78e8045134c3 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.240050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207afca6-334e-4d6e-9aec-78e8045134c3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.240706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207afca6-334e-4d6e-9aec-78e8045134c3 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.242723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 443/887] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:44.246929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:44.246929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.246929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.246929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.247425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] RBAC: Adding query filter params (interface=public, service_id=8c105766bda4410aa016f83d889eed7e, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:44.247425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:44.247425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:44.248601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702739cb-698d-4b7d-9551-d512944f6433 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.248897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702739cb-698d-4b7d-9551-d512944f6433 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.249185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702739cb-698d-4b7d-9551-d512944f6433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.256658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 445/888] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:56:44 2022] GET /identity/v3/endpoints?service_id=8c105766bda4410aa016f83d889eed7e&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:44.259801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:44.277730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-702739cb-698d-4b7d-9551-d512944f6433 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:44.279068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-702739cb-698d-4b7d-9551-d512944f6433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:44.369249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 444/889] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:44.394760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d56de8-e315-403a-bce0-2adc87d05fc9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:44.436788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c8308-8a92-449b-9336-d88a721f5cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:44.437778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c8308-8a92-449b-9336-d88a721f5cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.438266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c8308-8a92-449b-9336-d88a721f5cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.438796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c8308-8a92-449b-9336-d88a721f5cd3 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.439663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c8308-8a92-449b-9336-d88a721f5cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:44.441229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c8308-8a92-449b-9336-d88a721f5cd3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:44.448174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 446/890] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:56:44 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 15:56:44.650833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702739cb-698d-4b7d-9551-d512944f6433 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:44.652185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbece6-231c-4124-bcee-8832e85a0c0e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.653062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbece6-231c-4124-bcee-8832e85a0c0e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.653550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbece6-231c-4124-bcee-8832e85a0c0e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.655396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 447/891] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:44.660667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09a395-fe68-4d6e-ba7f-0bccd8428e53 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.661233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09a395-fe68-4d6e-ba7f-0bccd8428e53 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.661785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09a395-fe68-4d6e-ba7f-0bccd8428e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.682032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5d09a395-fe68-4d6e-ba7f-0bccd8428e53 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:44.683667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d09a395-fe68-4d6e-ba7f-0bccd8428e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:44.717371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:44.717545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:44.765971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 448/892] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:44.774049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702739cb-698d-4b7d-9551-d512944f6433 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:44.814041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:44.814930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.815367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.815788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.826880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:44.832159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:44.839990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:56:44.841045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:44.841952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 445/893] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Sep 4 15:56:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:44.844476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09a395-fe68-4d6e-ba7f-0bccd8428e53 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:44.948275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09a395-fe68-4d6e-ba7f-0bccd8428e53 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:44.983463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:44.984288 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:44.984731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:44.985155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:44.986006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:44.986456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:44.988014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:45.000825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 449/894] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Sep 4 15:56:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:45.002778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:45.053597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:45.073928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:45.126405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9bc2a-02a2-4a38-b38c-598a7dbd26d0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:45.143479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:45.164567 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0da4b9-e04b-4413-a1c7-02a05abfc06d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:45.170818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:45.171558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:45.172111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:45.172206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:45.185774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:45.187515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:45.205072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:45.206104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:45.207288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:45.208333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:45.208333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:45.208333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:45.209781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:45.210309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:45.210739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:45.211752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:45.212559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 446/895] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:45.213580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:45.216510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:45.306309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 450/896] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:56:45 2022] POST /identity/v3/services => generated 241 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:56:45.329071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c9d2-ed77-4a9f-92ce-8b3e68a91bdf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:45.377645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:45.378369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:45.378652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:45.378921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:45.379577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:45.379881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:45.381525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:45.400558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 447/897] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:45.402306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:45.493289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:45.523880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c474b5f-089e-4ee6-989b-c9c9cbd76158 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:45.559945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:45.560509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:45.560791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:45.561066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:45.570008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:45.571485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:45.584585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:45.585794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:45.587036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 451/898] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:56:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:45.593898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:45.617897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:45.705065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3562a-44ed-4bf7-9f6b-892fbe49f92b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:45.741496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:45.742145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:45.742426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:45.742697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:45.743433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:45.743740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:45.745166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:45.765580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 448/899] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Sep 4 15:56:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:45.774716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:45.882740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7748f-6dab-42e8-bf45-78e2365c6695 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:45.920508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff320d06-854e-471f-842c-7d9757a17302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:45.921518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff320d06-854e-471f-842c-7d9757a17302 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:45.922027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff320d06-854e-471f-842c-7d9757a17302 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:45.922514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff320d06-854e-471f-842c-7d9757a17302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:45.926688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:45.995840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff320d06-854e-471f-842c-7d9757a17302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:45.997801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff320d06-854e-471f-842c-7d9757a17302 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:46.105409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 452/900] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:56:46.107994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:46.191877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:46.214514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b897548-bfe8-41ae-a157-50f71b8e9962 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:46.250731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:46.251624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.251900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.252169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.252281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853b91-130b-4b7b-b7de-db30ab696b67 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.252480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853b91-130b-4b7b-b7de-db30ab696b67 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.252765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853b91-130b-4b7b-b7de-db30ab696b67 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.254411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 453/901] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:46.259822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0d5f6-8e30-47f0-afaa-f327a0ff1b35 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.260106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0d5f6-8e30-47f0-afaa-f327a0ff1b35 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.260381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0d5f6-8e30-47f0-afaa-f327a0ff1b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.262891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:46.264486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:46.279470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e3b0d5f6-8e30-47f0-afaa-f327a0ff1b35 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:46.280665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e3b0d5f6-8e30-47f0-afaa-f327a0ff1b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:46.281639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:46.282552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:46.283464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:46.283464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:46.283464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:46.285646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 449/902] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:46.289701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:46.327982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:46.377591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 454/903] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:46.425911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690413c5-4852-44e3-8a27-e5b11cb6f308 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:46.455931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:46.462229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:46.462852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.463129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.463402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.464062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:46.468042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:46.468042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:46.470976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c9702-98ee-4fcd-9319-f48a884914a6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.471269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c9702-98ee-4fcd-9319-f48a884914a6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.471550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c9702-98ee-4fcd-9319-f48a884914a6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.473154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 455/904] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:46.478930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678704c-01cf-4a62-b744-9a37edc3112c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.479241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678704c-01cf-4a62-b744-9a37edc3112c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.479606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678704c-01cf-4a62-b744-9a37edc3112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.479950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 450/905] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:46.489590 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:46.498463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e678704c-01cf-4a62-b744-9a37edc3112c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:46.499665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e678704c-01cf-4a62-b744-9a37edc3112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:46.581828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 456/906] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:46.592079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f3c97-c662-4a3b-95d4-960ca7c00289 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:46.627605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:46.628239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.628515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.628786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.635602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:46.637083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:46.649145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:46.650052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:46.650983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:46.655110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 451/907] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:46.656383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678704c-01cf-4a62-b744-9a37edc3112c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:46.680842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:46.731744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:46.763646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:46.769791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678704c-01cf-4a62-b744-9a37edc3112c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:46.784135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebfbf3-5a36-4837-9b89-fa4c966e7301 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:46.815842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:46.816374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.816635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.816921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.817951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:46.818499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:46.820504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:46.825308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:46.826209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:46.826667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:46.827117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:46.827872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:46.829390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:46.833300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 457/908] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:46.840161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:46.845633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:46.846554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:46.847811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:46.852425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 452/909] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Sep 4 15:56:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:46.853476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:46.953639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f718320-1515-4f81-9f5d-f7d93c46dbe7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:46.958587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b274b49-a4f3-414b-9bb2-3ec2ba379879 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:46.974184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:47.003254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.003467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.003569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.003657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.003741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.003843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.003927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.004009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.004091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] RBAC: Adding query filter params (role.id=4dc70a805edd4526856e1731a0cbbcaa, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=4fb881e220f041f1a7c99960be9aeb2c) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:47.004173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:47.004343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.004424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.004424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.004556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.004625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 453/910] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:47.005236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:47.011646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 458/911] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:46 2022] GET /identity/v3/role_assignments?user.id=4fb881e220f041f1a7c99960be9aeb2c&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=4dc70a805edd4526856e1731a0cbbcaa => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:47.012733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:47.046233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:47.135944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c48277-4be9-4a1c-9c5b-d22915386d3d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:47.165596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae5eb-98dc-406e-8b78-aa7d179fe7a8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:47.178907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.179552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.179830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.180099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.180793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.182335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:47.197504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:47.199302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:47.200138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 454/912] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:47.200953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:47.205117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.205797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.206155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.206438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.207096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:47.207389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.208772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.216024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 459/913] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:47.221988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:47.249061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:47.316672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:47.321408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0dbc4a-a994-4788-8ea4-621aad6b7ca5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:47.343125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58377f1e-4885-4f32-af24-b80c761141a2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:47.371084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.371866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.372252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.372629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.373367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:47.373801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.375272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.380848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 455/914] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:47.386732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:47.387283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.387903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.388186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.388464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.389177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] RBAC: Adding query filter params (interface=public, service_id=0870a4151c714991b27f597695b2fd01, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:47.389488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.392210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.398682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 460/915] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:56:47 2022] GET /identity/v3/endpoints?service_id=0870a4151c714991b27f597695b2fd01&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:47.404246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:47.509376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780fe57-4bc5-4723-bb2e-26077238b308 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:47.517259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:47.526798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c659e-0188-4416-8835-509719d59a13 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:47.548489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe544e96-7b89-4be6-96e6-280fdda45aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.549339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe544e96-7b89-4be6-96e6-280fdda45aac None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.549976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe544e96-7b89-4be6-96e6-280fdda45aac None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.550581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe544e96-7b89-4be6-96e6-280fdda45aac None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.551230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe544e96-7b89-4be6-96e6-280fdda45aac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:47.551568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe544e96-7b89-4be6-96e6-280fdda45aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.553681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe544e96-7b89-4be6-96e6-280fdda45aac None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.563217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 456/916] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Sep 4 15:56:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:47.563217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387b3d1-bc32-4172-9ab5-8259f0915f7a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.563217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387b3d1-bc32-4172-9ab5-8259f0915f7a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.563217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387b3d1-bc32-4172-9ab5-8259f0915f7a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.563217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 457/917] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:47.565700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca0200-97c0-45ba-92a3-5fe5a645400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.566331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca0200-97c0-45ba-92a3-5fe5a645400f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.566618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca0200-97c0-45ba-92a3-5fe5a645400f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.566941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca0200-97c0-45ba-92a3-5fe5a645400f None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.567039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387b3d1-bc32-4172-9ab5-8259f0915f7a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:47.567501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca0200-97c0-45ba-92a3-5fe5a645400f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.568969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca0200-97c0-45ba-92a3-5fe5a645400f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.576183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 461/918] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:56:47 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 15:56:47.577447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cc201-baab-4c13-8735-756e4a64d404 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.577783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cc201-baab-4c13-8735-756e4a64d404 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.578062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cc201-baab-4c13-8735-756e4a64d404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.599672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-eb7cc201-baab-4c13-8735-756e4a64d404 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:47.601282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eb7cc201-baab-4c13-8735-756e4a64d404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:47.635493 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:47.682972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 462/919] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:47.690747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387b3d1-bc32-4172-9ab5-8259f0915f7a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:47.732723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:47.733387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:47.733708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:47.733984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:47.734660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:47.736022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:47.831148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 458/920] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:56:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:47.965081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cc201-baab-4c13-8735-756e4a64d404 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:48.070905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cc201-baab-4c13-8735-756e4a64d404 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:48.117932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:48.118575 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:48.118856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:48.119131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:48.119776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:48.121159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:48.130872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:48.131908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:48.133172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 463/921] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:48.138650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:48.174015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:48.248044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc654b39-4f88-4471-b667-d563c57f1d8e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:48.285191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:48.285885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:48.286180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:48.286461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:48.287098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:48.287457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:48.288940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:48.294151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 459/922] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:48.299703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:48.303403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:48.412359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a873f7-3c75-4969-b70d-50509b50f534 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:48.452783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:48.453399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:48.453804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:48.454071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:48.454712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:48.455015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:48.456388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:48.461910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 464/923] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:48.467777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:48.571968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:48.582649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decda0f8-ba2c-42aa-8565-6c906a054796 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:48.622248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa5f03-d2cc-4096-a0a7-efbf68604beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:48.622862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5f03-d2cc-4096-a0a7-efbf68604beb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:48.623146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5f03-d2cc-4096-a0a7-efbf68604beb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:48.623420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa5f03-d2cc-4096-a0a7-efbf68604beb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:48.624123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa5f03-d2cc-4096-a0a7-efbf68604beb None admin] RBAC: Adding query filter params (interface=public, service_id=c56395f6649a418c966db8d05c6e6222, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:48.624431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa5f03-d2cc-4096-a0a7-efbf68604beb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:48.625824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa5f03-d2cc-4096-a0a7-efbf68604beb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:48.631553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 460/924] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:56:48 2022] GET /identity/v3/endpoints?service_id=c56395f6649a418c966db8d05c6e6222&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:48.637233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:48.707612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:48.748442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb012835-1a9d-42f7-9ee7-2393c8963708 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:48.785845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095493c-76e4-45da-bf97-28c70efb6ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:48.786463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095493c-76e4-45da-bf97-28c70efb6ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:48.786745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095493c-76e4-45da-bf97-28c70efb6ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:48.787019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095493c-76e4-45da-bf97-28c70efb6ee9 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:48.788237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095493c-76e4-45da-bf97-28c70efb6ee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:48.789620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095493c-76e4-45da-bf97-28c70efb6ee9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:48.796209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 465/925] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:56:48 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 15:56:49.141497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8338f2a-5b73-46bd-932b-4d5a3848d13d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:49.141873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8338f2a-5b73-46bd-932b-4d5a3848d13d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:49.142097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8338f2a-5b73-46bd-932b-4d5a3848d13d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:49.143696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 461/926] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:49.149341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de104-f391-45de-a7e8-a4b809061b4f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:49.149834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de104-f391-45de-a7e8-a4b809061b4f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:49.149913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de104-f391-45de-a7e8-a4b809061b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:49.170646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8b0de104-f391-45de-a7e8-a4b809061b4f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:49.171882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b0de104-f391-45de-a7e8-a4b809061b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:49.253087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 466/927] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:49.557120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8338f2a-5b73-46bd-932b-4d5a3848d13d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:49.611050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce77c98-172b-4d0b-bf46-aa030893cb02 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:49.611359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce77c98-172b-4d0b-bf46-aa030893cb02 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:49.611619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce77c98-172b-4d0b-bf46-aa030893cb02 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:49.613085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 467/928] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:49.620112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a76be5-4a85-4d2f-8062-a033113aab97 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:49.620420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a76be5-4a85-4d2f-8062-a033113aab97 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:49.620697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a76be5-4a85-4d2f-8062-a033113aab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:49.644474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-10a76be5-4a85-4d2f-8062-a033113aab97 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:49.646193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-10a76be5-4a85-4d2f-8062-a033113aab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:49.674032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8338f2a-5b73-46bd-932b-4d5a3848d13d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:49.719238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:49.721547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:49.722348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:49.722632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:49.723021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:49.734232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 468/929] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:49.736482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:49.743583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:49.744691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:49.745653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 462/930] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Sep 4 15:56:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:49.752853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a76be5-4a85-4d2f-8062-a033113aab97 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:49.863825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a76be5-4a85-4d2f-8062-a033113aab97 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:49.904229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:49.904857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:49.905141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:49.905417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:49.906122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:49.906435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:49.907812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:49.916608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 469/931] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Sep 4 15:56:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:49.922132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:49.941953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.011853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.035973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150ff3-8796-4394-b70e-9c603d603c8f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.071634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.072847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8f3ad-4c2b-4601-b1b8-dab71974af02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.073839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8f3ad-4c2b-4601-b1b8-dab71974af02 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.074178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8f3ad-4c2b-4601-b1b8-dab71974af02 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.074459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8f3ad-4c2b-4601-b1b8-dab71974af02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.086333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8f3ad-4c2b-4601-b1b8-dab71974af02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.087757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8f3ad-4c2b-4601-b1b8-dab71974af02 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-67a8f3ad-4c2b-4601-b1b8-dab71974af02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:50.103106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:50.104127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:50.105020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:50.105020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:50.105020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:50.105020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 463/932] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:50.106266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5e34e-8ecc-429c-89b9-a9b1fe0292f7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.106554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5e34e-8ecc-429c-89b9-a9b1fe0292f7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.106827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5e34e-8ecc-429c-89b9-a9b1fe0292f7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.108194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 464/933] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:50.112929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5e34e-8ecc-429c-89b9-a9b1fe0292f7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.128016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d607f-c1f2-4335-8f64-90b94ba384d2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.163012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed0c4b-7809-4c39-bf16-e3f05cdba731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.163817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed0c4b-7809-4c39-bf16-e3f05cdba731 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.164100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed0c4b-7809-4c39-bf16-e3f05cdba731 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.164371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed0c4b-7809-4c39-bf16-e3f05cdba731 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.165051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed0c4b-7809-4c39-bf16-e3f05cdba731 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.166454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed0c4b-7809-4c39-bf16-e3f05cdba731 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-10ed0c4b-7809-4c39-bf16-e3f05cdba731 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:50.175244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:50.176261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.177169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 470/934] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:50.178631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e46d5a-4302-4a4b-a640-f3c57f9b5ee5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.178631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e46d5a-4302-4a4b-a640-f3c57f9b5ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.178631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e46d5a-4302-4a4b-a640-f3c57f9b5ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.191372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.196911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-82e46d5a-4302-4a4b-a640-f3c57f9b5ee5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:50.198114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-82e46d5a-4302-4a4b-a640-f3c57f9b5ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:50.231657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5e34e-8ecc-429c-89b9-a9b1fe0292f7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.266269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 471/935] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:50.267111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e46d5a-4302-4a4b-a640-f3c57f9b5ee5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.274774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91853c-016d-41c9-a245-b74747403958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.275498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91853c-016d-41c9-a245-b74747403958 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.275863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91853c-016d-41c9-a245-b74747403958 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.276206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91853c-016d-41c9-a245-b74747403958 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.276968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91853c-016d-41c9-a245-b74747403958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:50.277346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91853c-016d-41c9-a245-b74747403958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.278786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91853c-016d-41c9-a245-b74747403958 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.293599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 465/936] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:50.298615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91853c-016d-41c9-a245-b74747403958 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.328089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.395039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e46d5a-4302-4a4b-a640-f3c57f9b5ee5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.433466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.434103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.434379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.434648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.435276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:50.435574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.436937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.446908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 472/937] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:50.448296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.456202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91853c-016d-41c9-a245-b74747403958 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.495509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.507870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.508979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.509417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.513487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.519883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.523014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:50.539000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:50.540229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:50.541432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:50.543079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 466/938] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:50.549748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.565251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d5c85-600e-4020-a41a-fc4d07a5e8e3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.576532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.604407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.605279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.605557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.605894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.606647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:50.607718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.609796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.616064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 473/939] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:50.616833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.660500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b9793-2972-4ece-8e10-cea4e01a0c1f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.693805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.694534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.694759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.694942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.695659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:50.695962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.697370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.716637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 467/940] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:50.717447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.734929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7c4b9-c502-4a22-bbcc-514f739a0034 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.766578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.774174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.775038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.775473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.775929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.776723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.778391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.780993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING keystone.server.flask.common [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 04 15:56:50.821985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e2a9-16cc-4f46-842e-ebc4f9e20ef9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.832356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.sql.core [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 04 15:56:50.832356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 04 15:56:50.832356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [parameters: {'id': '7cf9049c71554d78bb61e709b7660975', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 04 15:56:50.832356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81625) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7cf9049c71554d78bb61e709b7660975.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7cf9049c71554d78bb61e709b7660975. Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 15:56:50.843088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 04 15:56:50.844133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application yield resource Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.rollback() Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.force_reraise() Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise self.value Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application session.commit() Sep 04 15:56:50.845164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._prepare_impl() Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.flush() Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._flush(objects) Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application compat.raise_( Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application flush_context.execute() Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 04 15:56:50.846219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rec.execute(self) Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 04 15:56:50.847159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 15:56:50.847964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [parameters: {'id': '7cf9049c71554d78bb61e709b7660975', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.848874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:50.851563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7cf9049c71554d78bb61e709b7660975. Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 474/941] 10.222.0.39 () {60 vars in 1140 bytes} [Sun Sep 4 15:56:50 2022] POST /identity/v3/projects => generated 262 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:50.853199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:50.863365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825a25d-6ae4-4664-8686-d3e30d9b2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:50.869350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a25d-6ae4-4664-8686-d3e30d9b2041 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.869350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a25d-6ae4-4664-8686-d3e30d9b2041 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.869350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a25d-6ae4-4664-8686-d3e30d9b2041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.869350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825a25d-6ae4-4664-8686-d3e30d9b2041 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:50.869350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825a25d-6ae4-4664-8686-d3e30d9b2041 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:50.880468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:50.949651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198069d-be47-43b2-9c14-b367a32e006d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:50.952691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 468/942] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:56:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:50.954024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298ce19-94ec-489a-a3a7-d8373609ffdc None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:50.954316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298ce19-94ec-489a-a3a7-d8373609ffdc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:50.954592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298ce19-94ec-489a-a3a7-d8373609ffdc None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:50.957603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 469/943] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:50.957603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298ce19-94ec-489a-a3a7-d8373609ffdc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.009411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79020ab0-d61c-4e43-8407-ade065b081bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.010117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79020ab0-d61c-4e43-8407-ade065b081bf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.010404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79020ab0-d61c-4e43-8407-ade065b081bf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.010673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79020ab0-d61c-4e43-8407-ade065b081bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.023098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79020ab0-d61c-4e43-8407-ade065b081bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.024736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79020ab0-d61c-4e43-8407-ade065b081bf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-79020ab0-d61c-4e43-8407-ade065b081bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:51.042221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:51.043266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:51.049357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:51.049357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:51.049357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:51.049357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 475/944] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:51.049357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8dc1-38b0-4a8b-82e0-46fa4aa7fd05 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.049357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8dc1-38b0-4a8b-82e0-46fa4aa7fd05 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.049357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8dc1-38b0-4a8b-82e0-46fa4aa7fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.065895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fc6e8dc1-38b0-4a8b-82e0-46fa4aa7fd05 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:51.067487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fc6e8dc1-38b0-4a8b-82e0-46fa4aa7fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:51.089859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:51.093766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298ce19-94ec-489a-a3a7-d8373609ffdc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.134434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.135093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.135697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.135697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.143779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 476/945] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:51.144009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.144471 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8dc1-38b0-4a8b-82e0-46fa4aa7fd05 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.145515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.155277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:51.160869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:51.161845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:51.162792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:51.165742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 470/946] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:51.168722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.260733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8dc1-38b0-4a8b-82e0-46fa4aa7fd05 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.299523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.300143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.300476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.300758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.301512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:51.301850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.304156 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.306091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26779323-9ada-4536-bf4a-a3812525f180 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.316607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 477/947] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:51.327800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.353319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.354418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.354766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.355111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.355848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:51.356213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.358395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.385660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 471/948] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:51.416151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:51.433740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.487854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1e7a9-5403-44fe-9fcc-41d992ffbac2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.488023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:51.497131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.498165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.498629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.499153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.507511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.509481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:51.523216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:51.524159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:51.525064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:51.527534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 478/949] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:51.534194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.576731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b817ad-1174-49df-b3c9-a5f05e306e37 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.616109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.616791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.617146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.617519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.618150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.619648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:51.630364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:51.632151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:51.633414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:51.635682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 472/950] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:51.643120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.664559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede8549-d735-4edb-85c0-3993424b696e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.695570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:51.704900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.705887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.706171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.706443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.707133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:51.707445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.709117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.727553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 479/951] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:51.729843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.750754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:51.770014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ebccd-8f8c-47f3-bf8b-0a806d1d27f4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.816435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.817337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.817848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.819299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.820615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:51.821362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.823255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.835309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 473/952] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:51.836415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:51.849378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e258d0-50a4-4590-bfe8-f807e4fbb05b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.888446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:51.889173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:51.889461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:51.889811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/4fb881e220f041f1a7c99960be9aeb2c/roles/4dc70a805edd4526856e1731a0cbbcaa` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:51.892079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=4fb881e220f041f1a7c99960be9aeb2c, role_id=4dc70a805edd4526856e1731a0cbbcaa)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:51.893522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:51.959743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c226ed-b3ef-4771-92b4-afb329889f21 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:51.980454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 480/953] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:56:51 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/4fb881e220f041f1a7c99960be9aeb2c/roles/4dc70a805edd4526856e1731a0cbbcaa => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:56:52.007838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e94ff9-41ec-4b6a-b205-23b8d0c27380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:52.008530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e94ff9-41ec-4b6a-b205-23b8d0c27380 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:52.008968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e94ff9-41ec-4b6a-b205-23b8d0c27380 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:52.009209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e94ff9-41ec-4b6a-b205-23b8d0c27380 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:52.009905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e94ff9-41ec-4b6a-b205-23b8d0c27380 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:52.010270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e94ff9-41ec-4b6a-b205-23b8d0c27380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:52.011743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e94ff9-41ec-4b6a-b205-23b8d0c27380 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:52.018135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 474/954] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:52.024190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:52.083329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:52.140659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8711f58-213b-4204-8146-60da63f6f344 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:52.177431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f40d2-8413-41db-87e6-fb062502c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:52.178803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f40d2-8413-41db-87e6-fb062502c816 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:52.179090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f40d2-8413-41db-87e6-fb062502c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:52.179367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f40d2-8413-41db-87e6-fb062502c816 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:52.180050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f40d2-8413-41db-87e6-fb062502c816 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:52.181423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f40d2-8413-41db-87e6-fb062502c816 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:52.272098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 481/955] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:56:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:56:52.988713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b61fdd-e100-4b11-b1d7-26b90b36b3a7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:52.989275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b61fdd-e100-4b11-b1d7-26b90b36b3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:52.989275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b61fdd-e100-4b11-b1d7-26b90b36b3a7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:52.991846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 475/956] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:52.997232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceafd93-09c8-4913-87b3-ac65883e38aa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:52.997528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceafd93-09c8-4913-87b3-ac65883e38aa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:52.997841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceafd93-09c8-4913-87b3-ac65883e38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:53.017282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2ceafd93-09c8-4913-87b3-ac65883e38aa None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:53.018519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ceafd93-09c8-4913-87b3-ac65883e38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:53.096182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 482/957] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:53.446915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b61fdd-e100-4b11-b1d7-26b90b36b3a7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:53.475552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:53.595999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980c6d-da35-4875-a611-d9a779eb30bf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:53.596508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980c6d-da35-4875-a611-d9a779eb30bf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:53.596622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980c6d-da35-4875-a611-d9a779eb30bf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:53.598781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 483/958] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:53.605412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c85d1d-af99-4632-86a9-175152a807fb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:53.606007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c85d1d-af99-4632-86a9-175152a807fb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:53.606119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c85d1d-af99-4632-86a9-175152a807fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:53.626340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b61fdd-e100-4b11-b1d7-26b90b36b3a7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:53.632089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-61c85d1d-af99-4632-86a9-175152a807fb None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:53.632735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61c85d1d-af99-4632-86a9-175152a807fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:53.679234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:53.703050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:53.703695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:53.703971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:53.704242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:53.705091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:53.706234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:53.724924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:53.726395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:56:53.727375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:53.731140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 476/959] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:53.732477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:53.805688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 484/960] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:53.880433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:53.934865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983ca43-aae7-4dba-90ae-b9cee36c8ec6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:53.975333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7addf1-2772-49de-8d2f-a1877c80bcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:53.976297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7addf1-2772-49de-8d2f-a1877c80bcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:53.976752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7addf1-2772-49de-8d2f-a1877c80bcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:53.977285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7addf1-2772-49de-8d2f-a1877c80bcd6 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:53.978401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7addf1-2772-49de-8d2f-a1877c80bcd6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:53.978909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7addf1-2772-49de-8d2f-a1877c80bcd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:53.981093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7addf1-2772-49de-8d2f-a1877c80bcd6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:53.992252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 477/961] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:53.993740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c85d1d-af99-4632-86a9-175152a807fb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:54.115997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c85d1d-af99-4632-86a9-175152a807fb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:54.155855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:54.156825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.157314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.157878 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.158768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:54.159342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:54.160942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:54.164914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073fef2-af20-4465-aac5-612ebe835241 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.165428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073fef2-af20-4465-aac5-612ebe835241 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.165867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073fef2-af20-4465-aac5-612ebe835241 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.167620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 478/962] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:54.168932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 485/963] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:54.177346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fb5ff-306d-4b71-be55-4a41e6dafd1f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.177346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fb5ff-306d-4b71-be55-4a41e6dafd1f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.177346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fb5ff-306d-4b71-be55-4a41e6dafd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.192371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:54.199156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b50fb5ff-306d-4b71-be55-4a41e6dafd1f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:54.200859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b50fb5ff-306d-4b71-be55-4a41e6dafd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:54.254117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:54.301816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 479/964] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:54.338351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb51c73-ad72-498d-b15d-85469733a1b0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:54.377212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:54.377856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.378137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.378408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.379063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:54.380488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:54.430125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.sql.core [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 04 15:56:54.430125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 04 15:56:54.430125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [parameters: {'user_id': '2e125389bc234f82a5396e5c985f36e7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 04 15:56:54.430125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81625) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5fcbae41-f10c-4aa3-8256-9a779d201295 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Sep 04 15:56:54.441022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 15:56:54.441957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application yield resource Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.rollback() Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.force_reraise() Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise self.value Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application session.commit() Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 04 15:56:54.442908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._prepare_impl() Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.flush() Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._flush(objects) Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application compat.raise_( Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application flush_context.execute() Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rec.execute(self) Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 04 15:56:54.445705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 15:56:54.447774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 15:56:54.449819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [parameters: {'user_id': '2e125389bc234f82a5396e5c985f36e7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:54.453527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 04 15:56:54.455423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 04 15:56:54.459601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 04 15:56:54.459601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 04 15:56:54.459601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 04 15:56:54.459601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 04 15:56:54.459601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 04 15:56:54.459601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:54.459601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 486/965] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Sep 4 15:56:54 2022] POST /identity/v3/users => generated 160 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:54.460501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fb5ff-306d-4b71-be55-4a41e6dafd1f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:54.530203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:54.553750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f068e-4556-4797-8280-3b0a22d3ab0f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.554072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f068e-4556-4797-8280-3b0a22d3ab0f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.554313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f068e-4556-4797-8280-3b0a22d3ab0f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.562831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 487/966] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:54.564108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7bcb4-259d-4203-8fc7-abcb6ba90444 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.564395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7bcb4-259d-4203-8fc7-abcb6ba90444 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.564624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fb5ff-306d-4b71-be55-4a41e6dafd1f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:54.564728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7bcb4-259d-4203-8fc7-abcb6ba90444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.587279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3fa7bcb4-259d-4203-8fc7-abcb6ba90444 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:54.588842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3fa7bcb4-259d-4203-8fc7-abcb6ba90444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:54.616058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:54.616058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.616058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.616058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.625777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:54.627499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:54.642831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:54.645010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 480/967] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:56:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:54.647887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:54.673716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 488/968] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:54.674957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7bcb4-259d-4203-8fc7-abcb6ba90444 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:54.716739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:54.756510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5fdb3-3d04-4bf6-9409-75fae906d02c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:54.789415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:54.795824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:54.796712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.797188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.797700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.809294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:56:54.814573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:54.815500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:56:54.817038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:54.824427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7bcb4-259d-4203-8fc7-abcb6ba90444 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:54.824954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 481/969] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Sep 4 15:56:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:54.827033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:54.860517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:54.861143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:54.861746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:54.862024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:54.863372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:54.863685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:54.865880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:54.910841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 489/970] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Sep 4 15:56:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:54.957820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64031927-bcf8-4a9d-8370-6dd1e5a89e95 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:54.960165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.003543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.004171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.004395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.004637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.005337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:55.005676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.007474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.017202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 482/971] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Sep 4 15:56:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:55.024000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.051423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.079592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4bbf-8fa8-496c-93f7-8a8b9fa6cc56 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.091985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.118145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.118924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.119307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.119584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.120177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.121905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:55.134351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:55.135391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:55.136248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 490/972] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:55.141219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.146858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e7ad1-9df0-45b2-8b73-a3eac86dbd17 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.198719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.199680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.199970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.200248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.213146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.214608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:55.229317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:55.230346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:55.231367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:55.231367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:55.231367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:55.231367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 483/973] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:55.238966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.255522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e604f4-622f-4ef8-a914-6ec95188fbea None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.293384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.294296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.294575 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.294843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.295471 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:55.295775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.297147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.302311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 491/974] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:55.308205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.311057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.356554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d39e0-c7f6-4329-a305-d45c3e3c391b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.359196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.393332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.394225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.394673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.394955 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.395607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:55.395914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.398173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.409889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 484/975] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:55.415565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.431037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5be7c1-627b-4745-8443-dbce7050f5ae None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.464858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.465477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.465790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.466076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.466729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:55.467036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.468637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.473091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 492/976] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:55.479809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.528497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbc42c-b3c9-4820-9a3f-cf4d2d220b2d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.555429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.563496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.570294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.570294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.570294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.572261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.573727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:56:55.585296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:55.586285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:55.587259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 485/977] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:55.592060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.596312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36031ad3-4b2e-4511-b80e-cb22c5688d8b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.620065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.632616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.633251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.633497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.633797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.634504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] RBAC: Adding query filter params (interface=admin, service_id=c56395f6649a418c966db8d05c6e6222, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:55.634819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.636199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.641499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 493/978] 10.222.0.39 () {56 vars in 1249 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/endpoints?service_id=c56395f6649a418c966db8d05c6e6222&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:55.646991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.700171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47946c04-5a3d-4a44-89f6-7aee831acfc0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.736715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.737332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.737642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.737924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.738656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:55.739084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.741401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.749732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38b29f-5583-42f1-aa0a-3f72b2a4cd81 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.761734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 486/979] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:55.766646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:55.787964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.789201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.789846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.790132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.790409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:55.791017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:55.792382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:55.798363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 494/980] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 15:56:55.858878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:55.879674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d141de-f863-4f4d-9c50-7f68c93710ba None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:55.919065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc657d5-c5f0-452d-bb87-2900ab90a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:55.919691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc657d5-c5f0-452d-bb87-2900ab90a68b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:55.919971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc657d5-c5f0-452d-bb87-2900ab90a68b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:55.920242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc657d5-c5f0-452d-bb87-2900ab90a68b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.000164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc657d5-c5f0-452d-bb87-2900ab90a68b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:56.001903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc657d5-c5f0-452d-bb87-2900ab90a68b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:56.085365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:56.105267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 487/981] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:56:55 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 15:56:56.111247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:56.214112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe764398-8231-4b5b-96c2-617f11b18e18 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:56.227184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8dbdb-9420-4676-b312-cdf9218763bc None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.227569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8dbdb-9420-4676-b312-cdf9218763bc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.227918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8dbdb-9420-4676-b312-cdf9218763bc None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.229342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 488/982] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:56.234566 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9b4c-a633-4b75-931c-d76f18b9857b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.234858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9b4c-a633-4b75-931c-d76f18b9857b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.235140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9b4c-a633-4b75-931c-d76f18b9857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.252241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:56.252893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.253169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.253442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.259245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b86a9b4c-a633-4b75-931c-d76f18b9857b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:56.260432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b86a9b4c-a633-4b75-931c-d76f18b9857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:56.265298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:56.266802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:56:56.281826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:56.282793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:56:56.283989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:56:56.283989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:56:56.283989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:56.283989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 495/983] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:56.289019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:56.338492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 489/984] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:56.348940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:56.404122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c9b8d-b496-4ffa-973d-508a77565408 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:56.437929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:56.438542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.438826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.439105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.439749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:56.440056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:56.441428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:56.453602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 496/985] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:56.459693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9b4c-a633-4b75-931c-d76f18b9857b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:56.558050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9b4c-a633-4b75-931c-d76f18b9857b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:56.570847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:56.591956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:56.592529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.592810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.593083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.601334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:56.602800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:56:56.614768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:56.615654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:56:56.616597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:56.620826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 490/986] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:56.621105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:56.621513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:56.724965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b5b11-09ae-4611-ae93-7da3497ff61e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:56.727883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990cc1-2a30-4d2a-9bb3-21448faaa156 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:56.733405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:56.761924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd038060-e8d0-4077-a503-745a91a489bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:56.762752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd038060-e8d0-4077-a503-745a91a489bb None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.762944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd038060-e8d0-4077-a503-745a91a489bb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.763234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd038060-e8d0-4077-a503-745a91a489bb None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.763939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd038060-e8d0-4077-a503-745a91a489bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:56.765146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:56.765473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd038060-e8d0-4077-a503-745a91a489bb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:56.766319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.766793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.767282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.768169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:56.768664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:56.770386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:56.784242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 491/987] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:56.791500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:56.822713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:56.857688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 497/988] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:56:56 2022] POST /identity/v3/services => generated 242 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:56:56.901112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54bc12-ee9c-4a4b-a83c-1a5523d6c5d4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:56.951466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f072194-4ff0-4dbb-8437-dea7b88271f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:56.952142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072194-4ff0-4dbb-8437-dea7b88271f3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:56.952427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072194-4ff0-4dbb-8437-dea7b88271f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:56.952700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072194-4ff0-4dbb-8437-dea7b88271f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:56.953492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072194-4ff0-4dbb-8437-dea7b88271f3 None admin] RBAC: Adding query filter params (role.id=4dc70a805edd4526856e1731a0cbbcaa, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=4fb881e220f041f1a7c99960be9aeb2c) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:56.953827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072194-4ff0-4dbb-8437-dea7b88271f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:56.955265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072194-4ff0-4dbb-8437-dea7b88271f3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:56.962940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 492/989] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:56:56 2022] GET /identity/v3/role_assignments?user.id=4fb881e220f041f1a7c99960be9aeb2c&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=4dc70a805edd4526856e1731a0cbbcaa => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:57.762626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c878563-dea0-4bf6-87cb-b266f55b60fe None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:57.763139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c878563-dea0-4bf6-87cb-b266f55b60fe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:57.763235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c878563-dea0-4bf6-87cb-b266f55b60fe None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:57.764684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 498/990] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:57.769741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6c21-323c-40d0-8268-2868a5e7b26a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:57.770148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6c21-323c-40d0-8268-2868a5e7b26a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:57.770312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6c21-323c-40d0-8268-2868a5e7b26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:57.789426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-318d6c21-323c-40d0-8268-2868a5e7b26a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:57.790650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-318d6c21-323c-40d0-8268-2868a5e7b26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:57.859140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 493/991] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:58.129679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c878563-dea0-4bf6-87cb-b266f55b60fe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:58.231472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c878563-dea0-4bf6-87cb-b266f55b60fe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:58.265491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:58.266152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.266433 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.266707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.267290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:58.268659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:58.281979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:58.283071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 15:56:58.283940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:58.285893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 499/992] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Sep 4 15:56:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:58.289704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d6c21-323c-40d0-8268-2868a5e7b26a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:58.313495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:58.400713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d6c21-323c-40d0-8268-2868a5e7b26a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:58.435080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:58.435696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.435978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.436252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.436882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:58.437188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:58.438581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:58.444225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 494/993] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:58.449753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:58.481874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:58.554922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b63ae4-0215-43bd-ae5f-d988c7a08556 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:58.597079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d9e52-8627-43df-be99-293f333e5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:58.597910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d9e52-8627-43df-be99-293f333e5a7a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.598207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d9e52-8627-43df-be99-293f333e5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.598484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d9e52-8627-43df-be99-293f333e5a7a None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.599119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d9e52-8627-43df-be99-293f333e5a7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:58.599990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d9e52-8627-43df-be99-293f333e5a7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:58.601343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d9e52-8627-43df-be99-293f333e5a7a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:58.612864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 500/994] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Sep 4 15:56:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:58.613033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:58.707309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:58.723223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1e471-7e0c-4841-a5f6-96ab911e40df None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:58.760352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ed0cd-0aa6-4976-b798-5da092cabca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:58.760971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ed0cd-0aa6-4976-b798-5da092cabca9 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.761252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ed0cd-0aa6-4976-b798-5da092cabca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.761524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ed0cd-0aa6-4976-b798-5da092cabca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.762221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ed0cd-0aa6-4976-b798-5da092cabca9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:58.763583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ed0cd-0aa6-4976-b798-5da092cabca9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:58.785815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22c5b7-dd05-4aa1-b51b-13c2ed96cfc3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.786354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22c5b7-dd05-4aa1-b51b-13c2ed96cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.786354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22c5b7-dd05-4aa1-b51b-13c2ed96cfc3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.788527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 501/995] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:56:58.793327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470501-23c9-408c-ba65-e1daecbf2ef8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.793640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470501-23c9-408c-ba65-e1daecbf2ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.793923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470501-23c9-408c-ba65-e1daecbf2ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.812512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-88470501-23c9-408c-ba65-e1daecbf2ef8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:58.813751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-88470501-23c9-408c-ba65-e1daecbf2ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:58.845601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 495/996] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:56:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:56:58.909711 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 502/997] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:56:58.963535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626146a0-b799-4345-9ed5-025dc63f861e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.963771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626146a0-b799-4345-9ed5-025dc63f861e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.964050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626146a0-b799-4345-9ed5-025dc63f861e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.965610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 496/998] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:56:58.971185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976547-fbe9-4d96-9e05-7d07b67226c8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:58.971479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976547-fbe9-4d96-9e05-7d07b67226c8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:58.971816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976547-fbe9-4d96-9e05-7d07b67226c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:58.992327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fa976547-fbe9-4d96-9e05-7d07b67226c8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:56:58.993543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fa976547-fbe9-4d96-9e05-7d07b67226c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:56:59.009918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:59.064505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 503/999] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:56:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:56:59.182611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626146a0-b799-4345-9ed5-025dc63f861e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.288510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626146a0-b799-4345-9ed5-025dc63f861e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.327119 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.327755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.328033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.328303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.328886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.332386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:59.337337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa976547-fbe9-4d96-9e05-7d07b67226c8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:59.341391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:59.342412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:56:59.348063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 497/1000] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:56:59.350467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.365150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:59.452440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa976547-fbe9-4d96-9e05-7d07b67226c8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.457323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4bb7e-5903-4adb-87e9-478a38dd6158 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.488595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.489226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.489504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.489808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.490949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.492314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:59.496348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.496952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.497231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.497503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.498149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] RBAC: Adding query filter params (name=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:59.498452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.499786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:56:59.501293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:56:59.502310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:56:59.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 504/1001] 10.222.0.39 () {56 vars in 1105 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:56:59.512540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 498/1002] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:59.512801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.513817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.529042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:59.594518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:59.619406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce328b-267d-4d40-ac15-42cf152d0efe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.619926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67541dbf-124c-4cfc-a3cb-f0791f3b40fb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.655322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.655925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.656206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.656476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.657095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:59.657394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.658801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:59.662518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.663296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.663725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.663832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 505/1003] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:59.664239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.665077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:59.665600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.667230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:59.669496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.674328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 499/1004] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:56:59.680616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.765055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:56:59.786725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9fac3-291b-4b51-928f-040d4c5dbed3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.787839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85428-c17e-4d8f-869d-240a64a925f6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.821886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.822548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.822832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.823106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.823730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:59.824030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.824870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53fd76-019e-4c3a-9db2-a4c329838f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.825399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:59.825503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53fd76-019e-4c3a-9db2-a4c329838f93 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.825783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53fd76-019e-4c3a-9db2-a4c329838f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.826060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53fd76-019e-4c3a-9db2-a4c329838f93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.826884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53fd76-019e-4c3a-9db2-a4c329838f93 None admin] RBAC: Adding query filter params (interface=public, service_id=556f6adfc9994639ad3e7c56b40279a2, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:56:59.827193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53fd76-019e-4c3a-9db2-a4c329838f93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.828772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53fd76-019e-4c3a-9db2-a4c329838f93 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:56:59.830173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 506/1005] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:59.839535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 500/1006] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/endpoints?service_id=556f6adfc9994639ad3e7c56b40279a2&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:56:59.843880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:56:59.954056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62fe5-dee9-4ca3-a2c3-08b02a860e62 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:56:59.992403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e9032-c79c-42c3-8684-eb07d04b7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:56:59.993021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9032-c79c-42c3-8684-eb07d04b7b23 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:56:59.993305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9032-c79c-42c3-8684-eb07d04b7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:56:59.993656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9032-c79c-42c3-8684-eb07d04b7b23 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:56:59.994222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e9032-c79c-42c3-8684-eb07d04b7b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:56:59.995598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e9032-c79c-42c3-8684-eb07d04b7b23 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:00.002577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 507/1007] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:56:59 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 15:57:00.813140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17c5af-107f-4c7f-ab8b-a7f018c70d92 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:00.813709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17c5af-107f-4c7f-ab8b-a7f018c70d92 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:00.813709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17c5af-107f-4c7f-ab8b-a7f018c70d92 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:00.815219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 501/1008] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:00.820374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0647d01-57b9-4cfb-ba6c-58b41dba5105 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:00.820833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0647d01-57b9-4cfb-ba6c-58b41dba5105 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:00.821002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0647d01-57b9-4cfb-ba6c-58b41dba5105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:00.834066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:00.840061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e0647d01-57b9-4cfb-ba6c-58b41dba5105 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:00.841257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e0647d01-57b9-4cfb-ba6c-58b41dba5105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:00.911503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 508/1009] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:01.194594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17c5af-107f-4c7f-ab8b-a7f018c70d92 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:01.197520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:01.300127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17c5af-107f-4c7f-ab8b-a7f018c70d92 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:01.344704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c562d-d82d-44d6-b6a6-1cebfabc65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:01.345752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c562d-d82d-44d6-b6a6-1cebfabc65f5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:01.346235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c562d-d82d-44d6-b6a6-1cebfabc65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:01.346706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c562d-d82d-44d6-b6a6-1cebfabc65f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:01.348034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c562d-d82d-44d6-b6a6-1cebfabc65f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:01.349727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c562d-d82d-44d6-b6a6-1cebfabc65f5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:01.351851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 502/1010] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:01.357189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0647d01-57b9-4cfb-ba6c-58b41dba5105 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:01.458934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0647d01-57b9-4cfb-ba6c-58b41dba5105 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:01.493041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7968f96-62f8-4917-8a2a-34972c6108c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:01.493691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7968f96-62f8-4917-8a2a-34972c6108c1 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:01.493976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7968f96-62f8-4917-8a2a-34972c6108c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:01.494250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7968f96-62f8-4917-8a2a-34972c6108c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:01.494893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7968f96-62f8-4917-8a2a-34972c6108c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:01.496325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7968f96-62f8-4917-8a2a-34972c6108c1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:01.502043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:01.610691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 509/1011] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:57:01 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:01.793016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54a870-8863-4378-ad86-182337256d8d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:01.793222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54a870-8863-4378-ad86-182337256d8d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:01.793503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54a870-8863-4378-ad86-182337256d8d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:01.794880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 503/1012] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:01.800368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317783-1ef5-429e-aac0-07ce08d78bd7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:01.800690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317783-1ef5-429e-aac0-07ce08d78bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:01.800970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317783-1ef5-429e-aac0-07ce08d78bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:01.820837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-99317783-1ef5-429e-aac0-07ce08d78bd7 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:01.822082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-99317783-1ef5-429e-aac0-07ce08d78bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:01.891702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 510/1013] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:01.923514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c0668-7229-43a0-968f-44472e69c56a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:01.924036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c0668-7229-43a0-968f-44472e69c56a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:01.924116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c0668-7229-43a0-968f-44472e69c56a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:01.925557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 504/1014] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:01.930741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19fdb-de9e-4727-aa02-11474e012b6b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:01.931099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19fdb-de9e-4727-aa02-11474e012b6b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:01.931383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19fdb-de9e-4727-aa02-11474e012b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:01.949941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a3a19fdb-de9e-4727-aa02-11474e012b6b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:01.951142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a3a19fdb-de9e-4727-aa02-11474e012b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:02.022963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 511/1015] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:02.167580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c0668-7229-43a0-968f-44472e69c56a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:02.238330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:02.270666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c0668-7229-43a0-968f-44472e69c56a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:02.300100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a19fdb-de9e-4727-aa02-11474e012b6b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:02.314771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:02.315383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:02.315669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:02.315948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:02.316996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:02.318584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:02.355004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:02.394642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 505/1016] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:57:02 2022] POST /identity/v3/services => generated 242 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:02.408874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a19fdb-de9e-4727-aa02-11474e012b6b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:02.457657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:02.459190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:02.459669 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:02.460029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:02.460603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:02.462105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:02.473038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:02.477843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:02.478820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 512/1017] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:57:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:02.481366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:02.590494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d40fe-d3b5-4317-87b5-365e656463f9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:02.621844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:02.628385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:02.634845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:02.634845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:02.634845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:02.634845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:02.634845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:02.634845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:02.637707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 506/1018] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:57:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:02.643234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:02.750685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16ef3-7e20-4a94-bfe4-ff2f87916a19 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:02.761651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:02.789774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b153d-0d66-4a1f-bd18-3d67d29eb931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:02.790391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b153d-0d66-4a1f-bd18-3d67d29eb931 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:02.790679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b153d-0d66-4a1f-bd18-3d67d29eb931 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:02.790958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b153d-0d66-4a1f-bd18-3d67d29eb931 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:02.791592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b153d-0d66-4a1f-bd18-3d67d29eb931 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:02.791903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b153d-0d66-4a1f-bd18-3d67d29eb931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:02.793275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b153d-0d66-4a1f-bd18-3d67d29eb931 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:02.802981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 513/1019] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Sep 4 15:57:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:02.804188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:02.908960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe88d28-dd63-4cf0-a079-7f9750541e0b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:02.942585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0653b-6fb7-4348-9731-ae30dd3c5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:02.943820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0653b-6fb7-4348-9731-ae30dd3c5670 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:02.944071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0653b-6fb7-4348-9731-ae30dd3c5670 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:02.944451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0653b-6fb7-4348-9731-ae30dd3c5670 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:02.945141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0653b-6fb7-4348-9731-ae30dd3c5670 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:02.946532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0653b-6fb7-4348-9731-ae30dd3c5670 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:03.033637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 507/1020] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:57:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:03.604125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5bac9-60db-436f-b100-d8bcb6b914d5 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:03.604633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5bac9-60db-436f-b100-d8bcb6b914d5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:03.604633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5bac9-60db-436f-b100-d8bcb6b914d5 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:03.606114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 514/1021] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:03.611157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ab05-1bdc-40ac-8bad-19dafdaeae93 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:03.611451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ab05-1bdc-40ac-8bad-19dafdaeae93 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:03.611740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ab05-1bdc-40ac-8bad-19dafdaeae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:03.631203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3506ab05-1bdc-40ac-8bad-19dafdaeae93 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:03.632379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3506ab05-1bdc-40ac-8bad-19dafdaeae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:03.679681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:03.706347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 508/1022] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:03.986559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5bac9-60db-436f-b100-d8bcb6b914d5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:04.091645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5bac9-60db-436f-b100-d8bcb6b914d5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:04.126530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba47e5-9fb2-4ede-b351-8acb0a492827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:04.127168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba47e5-9fb2-4ede-b351-8acb0a492827 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:04.127448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba47e5-9fb2-4ede-b351-8acb0a492827 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:04.127723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba47e5-9fb2-4ede-b351-8acb0a492827 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:04.128832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba47e5-9fb2-4ede-b351-8acb0a492827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:04.130326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba47e5-9fb2-4ede-b351-8acb0a492827 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:04.132611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 515/1023] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:04.138137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506ab05-1bdc-40ac-8bad-19dafdaeae93 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:04.237797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506ab05-1bdc-40ac-8bad-19dafdaeae93 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:04.272182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba718654-234c-4392-b1ed-7d467c5759d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:04.272960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba718654-234c-4392-b1ed-7d467c5759d3 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:04.273240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba718654-234c-4392-b1ed-7d467c5759d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:04.273510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba718654-234c-4392-b1ed-7d467c5759d3 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:04.274196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba718654-234c-4392-b1ed-7d467c5759d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:04.275623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba718654-234c-4392-b1ed-7d467c5759d3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:04.358813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 509/1024] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:57:04 2022] POST /identity/v3/users => generated 285 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:04.360923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e81383-23fc-472e-81c0-a14cd30150dc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:04.361218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e81383-23fc-472e-81c0-a14cd30150dc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:04.361502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e81383-23fc-472e-81c0-a14cd30150dc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:04.363139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 516/1025] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:04.368884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7672-7bce-4639-a6e7-a4c86329939e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:04.369183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7672-7bce-4639-a6e7-a4c86329939e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:04.369452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7672-7bce-4639-a6e7-a4c86329939e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:04.391492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-eefd7672-7bce-4639-a6e7-a4c86329939e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:04.392730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eefd7672-7bce-4639-a6e7-a4c86329939e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:04.424209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:04.472243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 510/1026] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:04.749512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e81383-23fc-472e-81c0-a14cd30150dc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:04.795695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:04.857818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e81383-23fc-472e-81c0-a14cd30150dc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:04.894776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e568dc-9a60-46df-83bf-659fdb90a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:04.895407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e568dc-9a60-46df-83bf-659fdb90a65d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:04.895690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e568dc-9a60-46df-83bf-659fdb90a65d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:04.895965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e568dc-9a60-46df-83bf-659fdb90a65d None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:04.897011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e568dc-9a60-46df-83bf-659fdb90a65d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:04.898456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e568dc-9a60-46df-83bf-659fdb90a65d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-63e568dc-9a60-46df-83bf-659fdb90a65d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:04.908671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:04.909797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:57:04.910642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:04.921597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 517/1027] 10.222.0.39 () {56 vars in 1105 bytes} [Sun Sep 4 15:57:04 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:04.921769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7672-7bce-4639-a6e7-a4c86329939e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:05.029400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7672-7bce-4639-a6e7-a4c86329939e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:05.064698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e60f7a-05e1-4ef4-ba4b-5563bd2b59df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:05.065311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e60f7a-05e1-4ef4-ba4b-5563bd2b59df None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.065613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e60f7a-05e1-4ef4-ba4b-5563bd2b59df None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.065891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e60f7a-05e1-4ef4-ba4b-5563bd2b59df None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.066521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e60f7a-05e1-4ef4-ba4b-5563bd2b59df None admin] RBAC: Adding query filter params (name=image) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:05.066825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e60f7a-05e1-4ef4-ba4b-5563bd2b59df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:05.068212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e60f7a-05e1-4ef4-ba4b-5563bd2b59df None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:05.072086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9059321-5fb4-43d6-b2df-fa37ec26f4ac None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.072370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9059321-5fb4-43d6-b2df-fa37ec26f4ac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.072639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9059321-5fb4-43d6-b2df-fa37ec26f4ac None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.078134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 511/1028] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:05.078280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 518/1029] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:57:05.078839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9059321-5fb4-43d6-b2df-fa37ec26f4ac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:05.079222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee8476-fb0f-48bd-842d-29c74ea1b2d3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.079511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee8476-fb0f-48bd-842d-29c74ea1b2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.079784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee8476-fb0f-48bd-842d-29c74ea1b2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.097930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ceee8476-fb0f-48bd-842d-29c74ea1b2d3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:05.099107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ceee8476-fb0f-48bd-842d-29c74ea1b2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:05.120503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:05.173920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 512/1030] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:05.186048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9059321-5fb4-43d6-b2df-fa37ec26f4ac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:05.189367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:05.223057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:05.223959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.224920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.225353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.226167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:05.226621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:05.228205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:05.237719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 519/1031] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:05.239620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee8476-fb0f-48bd-842d-29c74ea1b2d3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:05.347007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee8476-fb0f-48bd-842d-29c74ea1b2d3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:05.382578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:05.383177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.383454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.383726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.384420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] RBAC: Adding query filter params (interface=public, service_id=c0cfff6f02074a8fa78c6efbfc53a769, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:05.384724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:05.386154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:05.395939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 513/1032] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:57:05 2022] GET /identity/v3/endpoints?service_id=c0cfff6f02074a8fa78c6efbfc53a769&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:05.397166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:05.448980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:05.471696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:05.500589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fec28-14fb-4887-8fcd-626feb31bf11 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:05.540742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:05.541352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.541657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.541936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.542957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:05.544321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:05.551046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 520/1033] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:57:05 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 15:57:05.555056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a1d6c-d943-457e-be20-d02e97ad4bea None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:05.594014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:05.594579 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.594856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.595125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.607912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:05.612871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:05.613917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:05.614863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 514/1034] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Sep 4 15:57:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:05.620127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:05.654688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:05.727015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c549-160e-4a44-ba7a-8ad8d0e98078 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:05.762104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:05.762768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.763016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.763254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.763913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:05.764209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:05.765653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:05.774944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 521/1035] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Sep 4 15:57:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:05.780422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:05.862675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:05.882701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296aa3c-1fce-4e4f-9401-0ce16211b129 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:05.917333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:05.917980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:05.918255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:05.918529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:05.928700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:05.930198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:05.945405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:05.946550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:57:05.947441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:05.947441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:05.947441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:05.949627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 515/1036] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:05.952447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:06.049994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:06.057802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14475021-9adb-4859-a3e8-bcefabb65f45 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:06.091998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:06.092606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.092886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.093157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.093818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:06.094120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:06.095481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:06.112548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 522/1037] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:06.114322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:06.215562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3e56f-958c-4f52-801a-a025e91e1959 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:06.245137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:06.250609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6d239-a764-4d87-aea5-6da3dddce24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:06.251488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d239-a764-4d87-aea5-6da3dddce24b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.251763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d239-a764-4d87-aea5-6da3dddce24b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.252030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d239-a764-4d87-aea5-6da3dddce24b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.260244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6d239-a764-4d87-aea5-6da3dddce24b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:06.261687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6d239-a764-4d87-aea5-6da3dddce24b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c9f6d239-a764-4d87-aea5-6da3dddce24b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:57:06.272880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:06.273860 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:06.275386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 516/1038] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:57:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:06.279752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:06.363681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251b342-dc06-403b-bf85-a6b21ef19ad1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.363960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251b342-dc06-403b-bf85-a6b21ef19ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.364232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251b342-dc06-403b-bf85-a6b21ef19ad1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.366550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 517/1039] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:06.371560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395db011-1ad4-476e-bdf8-7967afe3c269 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.371841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395db011-1ad4-476e-bdf8-7967afe3c269 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.372108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395db011-1ad4-476e-bdf8-7967afe3c269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.388006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f3b9-ce34-4278-9fd4-9236f7a3770d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:06.393206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-395db011-1ad4-476e-bdf8-7967afe3c269 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:06.394575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-395db011-1ad4-476e-bdf8-7967afe3c269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:06.426333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:06.426944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.427749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.428254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.428994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:06.429299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:06.430787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:06.448694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 523/1040] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:57:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:06.454164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:06.469286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 518/1041] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:57:06.469732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:06.560871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea89b-a824-41e7-8df9-8147003d783f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:06.595699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:06.596310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.596592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.596872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.669039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:06.670444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:06.684559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:06.741382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395db011-1ad4-476e-bdf8-7967afe3c269 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:06.773911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 524/1042] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 15:57:06.779509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:06.845869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395db011-1ad4-476e-bdf8-7967afe3c269 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:06.869368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:06.882328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:06.882959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.883638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.883638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.890880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ddf05-3fd8-4825-8add-d93aa755047d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:06.893909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:06.899040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:06.900288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:06.901147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 519/1043] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:06.906147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:06.908765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:06.930934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:06.931498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:06.931785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:06.932064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:06.942335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:06.943841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:06.959186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:06.960819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:57:06.962103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:06.962103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:06.962103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:06.962103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 525/1044] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:06.965935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.025895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe7652-9c5a-4c32-b3e9-6927358531d6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.063414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.064366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.064869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.065318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.066209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:07.066704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.069139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.077389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e5a6c-a837-4688-b892-4695d29332ca None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.081739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 520/1045] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:07.084220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.117548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.118186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.118521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.118800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.119550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:07.120482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.121353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.131669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:07.134477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 526/1046] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:07.140568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.169209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:07.199448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56138e-ce36-4827-9af8-92e9a8c10439 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.236723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.237477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.237762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.238021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] PATH_INFO: `/v3/users/f53672a05a434d3cb5b034820cd90ecc` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.251741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e901b3e-e15b-499f-bc62-e00c2ae64947 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.252215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] RBAC: Authorizing `identity:get_user(user_id=f53672a05a434d3cb5b034820cd90ecc)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.253692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.255568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 521/1047] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:07 2022] GET /identity/v3/users/f53672a05a434d3cb5b034820cd90ecc => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:07.261130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.289970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.290608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.290890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.291173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.297931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.299382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:57:07.311798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:07.312700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:07.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 527/1048] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:57:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:07.320929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.370838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7692c-df96-4203-bda1-214a13c080cb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.372408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:07.405938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ea709-06e0-4c14-8bc2-baaba242a1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.406543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ea709-06e0-4c14-8bc2-baaba242a1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.406820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ea709-06e0-4c14-8bc2-baaba242a1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.407089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ea709-06e0-4c14-8bc2-baaba242a1a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.416098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:07.434660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989d8a-7d95-4491-9418-faeed469bd70 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.472449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.473046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.473252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.473520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.475041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:07.475355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.476778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.480524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ea709-06e0-4c14-8bc2-baaba242a1a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.482037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ea709-06e0-4c14-8bc2-baaba242a1a0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.494228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 528/1049] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:57:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:07.495470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.584326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 522/1050] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 15:57:07.585609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76be7bf-1385-4826-a54a-0817774348ef None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.585902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76be7bf-1385-4826-a54a-0817774348ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.586176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76be7bf-1385-4826-a54a-0817774348ef None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.589605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 523/1051] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:07.590330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76be7bf-1385-4826-a54a-0817774348ef None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.592977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:07.596691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a275-603c-4a89-91bf-bcb9bac7ec69 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.632958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c5933-e6fb-4771-a62f-0e1d4669f428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.633688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5933-e6fb-4771-a62f-0e1d4669f428 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.633975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5933-e6fb-4771-a62f-0e1d4669f428 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.634250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5933-e6fb-4771-a62f-0e1d4669f428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.634968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c5933-e6fb-4771-a62f-0e1d4669f428 None admin] RBAC: Adding query filter params (role.id=4dc70a805edd4526856e1731a0cbbcaa, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=46fd540df66143b2b768cc8d9f4beed7) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:07.635277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c5933-e6fb-4771-a62f-0e1d4669f428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.636706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c5933-e6fb-4771-a62f-0e1d4669f428 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.642090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 529/1052] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:07 2022] GET /identity/v3/role_assignments?user.id=46fd540df66143b2b768cc8d9f4beed7&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=4dc70a805edd4526856e1731a0cbbcaa => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:07.643069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bf3a-9470-4b4a-b1f2-2d00a162fad4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.643357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bf3a-9470-4b4a-b1f2-2d00a162fad4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.643632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bf3a-9470-4b4a-b1f2-2d00a162fad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.671311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8162bf3a-9470-4b4a-b1f2-2d00a162fad4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:07.672561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8162bf3a-9470-4b4a-b1f2-2d00a162fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:07.675076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:07.703150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76be7bf-1385-4826-a54a-0817774348ef None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.745581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.746161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.746448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.746730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.748014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] RBAC: Authorizing `identity:get_project(project_id=7636c6be385f4ea08420af6bfbade511)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.749455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.751769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 524/1053] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:07 2022] GET /identity/v3/projects/7636c6be385f4ea08420af6bfbade511 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:07.755075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 530/1054] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:07.758662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:07.833012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:07.864819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18026826-489a-4604-a507-dacc11bfbeaa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:07.900261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:07.901100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:07.901558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:07.902019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:07.902890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=7636c6be385f4ea08420af6bfbade511, user.id=f53672a05a434d3cb5b034820cd90ecc) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:07.903442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:07.905013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:07.911003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 525/1055] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:07 2022] GET /identity/v3/role_assignments?user.id=f53672a05a434d3cb5b034820cd90ecc&scope.project.id=7636c6be385f4ea08420af6bfbade511&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:08.043449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162bf3a-9470-4b4a-b1f2-2d00a162fad4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:08.156434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162bf3a-9470-4b4a-b1f2-2d00a162fad4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:08.190343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:08.190979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:08.191258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:08.191531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:08.192111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:08.194062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:08.202873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:08.204066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:08.204943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 531/1056] 10.222.0.39 () {56 vars in 1105 bytes} [Sun Sep 4 15:57:08 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:08.210022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:08.316552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de0e6-d3a5-489a-8e9b-82b0f66a43b4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:08.355138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:08.355752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:08.356036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:08.356310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:08.356940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] RBAC: Adding query filter params (name=image) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:08.357245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:08.358634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:08.359695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:08.368747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 526/1057] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:08.370060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:08.372987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:08.470998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a097-752d-4178-aaeb-a4d963091551 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:08.504341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc65c1-e4ab-477e-9529-a17fa8c66982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:08.505008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc65c1-e4ab-477e-9529-a17fa8c66982 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:08.505319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc65c1-e4ab-477e-9529-a17fa8c66982 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:08.505837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc65c1-e4ab-477e-9529-a17fa8c66982 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:08.506477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc65c1-e4ab-477e-9529-a17fa8c66982 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:08.506776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc65c1-e4ab-477e-9529-a17fa8c66982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:08.508133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc65c1-e4ab-477e-9529-a17fa8c66982 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:08.513171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 532/1058] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:08.518645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:08.627864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c4fc-0a5d-4596-9726-4208c8abcc04 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:08.663265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afa859-788d-4ffd-8a4d-c900493b10a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:08.663854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa859-788d-4ffd-8a4d-c900493b10a6 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:08.664131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa859-788d-4ffd-8a4d-c900493b10a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:08.664399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa859-788d-4ffd-8a4d-c900493b10a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:08.665058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afa859-788d-4ffd-8a4d-c900493b10a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:08.666433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afa859-788d-4ffd-8a4d-c900493b10a6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:08.830577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 527/1059] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:57:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 15:57:09.581024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49162cc1-406f-4a42-8eaa-3e05b489441a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:09.581534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49162cc1-406f-4a42-8eaa-3e05b489441a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:09.581665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49162cc1-406f-4a42-8eaa-3e05b489441a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:09.583103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 533/1060] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:09.588536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2612-01d0-4268-ac2e-5b211195d097 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:09.588914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2612-01d0-4268-ac2e-5b211195d097 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:09.589087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2612-01d0-4268-ac2e-5b211195d097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:09.608012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dceb2612-01d0-4268-ac2e-5b211195d097 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:09.609203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dceb2612-01d0-4268-ac2e-5b211195d097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:09.663934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:09.687465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 528/1061] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:09.812759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5d3de-6809-4fcb-8b3b-1ad4b38e0df7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:09.813081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5d3de-6809-4fcb-8b3b-1ad4b38e0df7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:09.813315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5d3de-6809-4fcb-8b3b-1ad4b38e0df7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:09.814879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 534/1062] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 15:57:09.820762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262f13-be5f-428e-af7f-0f0b7f3c6fdd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:09.821054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262f13-be5f-428e-af7f-0f0b7f3c6fdd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:09.821332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262f13-be5f-428e-af7f-0f0b7f3c6fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:09.840512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b0262f13-be5f-428e-af7f-0f0b7f3c6fdd None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:09.842014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b0262f13-be5f-428e-af7f-0f0b7f3c6fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:09.913598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 529/1063] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:57:09.973190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5d3de-6809-4fcb-8b3b-1ad4b38e0df7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.042576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:10.080310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5d3de-6809-4fcb-8b3b-1ad4b38e0df7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.114480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.115859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.116244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.116508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.126848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:10.133065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:10.134012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:57:10.134868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:10.137878 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 535/1064] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Sep 4 15:57:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:10.140074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0262f13-be5f-428e-af7f-0f0b7f3c6fdd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.188761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.246791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0262f13-be5f-428e-af7f-0f0b7f3c6fdd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.259188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:10.282156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.282816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.283146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.283382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.284043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:10.284341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.285717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:10.295604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 530/1065] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:10.300809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.303690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd315ad-89f5-4394-b57d-20b55b077daa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.345097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:10.355474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.356110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.363860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.363860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.367534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:10.372349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:10.373280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:10.374520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:10.378693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 536/1066] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:10.379924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.413673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2334e-0b53-4b3a-9937-950987a9c270 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.450589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.451224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.451507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.451780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.463242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.464692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:10.479297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:10.480297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:57:10.481205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:10.481205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:10.481205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:10.481205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 531/1067] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:10.486142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.487256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970544eb-deea-4a63-b53c-4803310c3c2f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.514651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:10.526102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.526909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.527340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.527759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.529420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:10.529420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.530670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:10.541172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 537/1068] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:10.546764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.593779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9c3a-008c-49bc-b056-ccb762e370dc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.598809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:10.630883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.631424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.631710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.631979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.632622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:10.632925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.634309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:10.651941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 532/1069] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:10.652673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.657860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faedefc-da13-40da-a8a7-068400d982c5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.695533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.696533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.696823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.697413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] PATH_INFO: `/v3/users/f53672a05a434d3cb5b034820cd90ecc` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.698602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] RBAC: Authorizing `identity:get_user(user_id=f53672a05a434d3cb5b034820cd90ecc)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.700053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:10.706514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 538/1070] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/users/f53672a05a434d3cb5b034820cd90ecc => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:10.707350 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.742619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:10.758908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ea786-50ff-4b53-a5a8-0325884e67d6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.796432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbb2ad-4e04-4bbc-af47-96ea0c04becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.797337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb2ad-4e04-4bbc-af47-96ea0c04becb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.797652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb2ad-4e04-4bbc-af47-96ea0c04becb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.797926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb2ad-4e04-4bbc-af47-96ea0c04becb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.808928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbb2ad-4e04-4bbc-af47-96ea0c04becb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.811120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbb2ad-4e04-4bbc-af47-96ea0c04becb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:10.818756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98246aa5-5748-42ac-857e-38efa4ef0659 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ddbbb2ad-4e04-4bbc-af47-96ea0c04becb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:57:10.825903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:10.826898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 533/1071] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61847b-044c-439e-abdb-4848b3d2bcb4 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.828668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61847b-044c-439e-abdb-4848b3d2bcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.829396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61847b-044c-439e-abdb-4848b3d2bcb4 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.830174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 534/1072] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:10.833541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61847b-044c-439e-abdb-4848b3d2bcb4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.838955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:10.857233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287ff86-cb92-490c-bb95-f1a6f74d14fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.857903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ff86-cb92-490c-bb95-f1a6f74d14fb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.858187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ff86-cb92-490c-bb95-f1a6f74d14fb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.858467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ff86-cb92-490c-bb95-f1a6f74d14fb None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.859595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287ff86-cb92-490c-bb95-f1a6f74d14fb None admin] RBAC: Authorizing `identity:get_project(project_id=7636c6be385f4ea08420af6bfbade511)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.861027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287ff86-cb92-490c-bb95-f1a6f74d14fb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:10.862701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 539/1073] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/projects/7636c6be385f4ea08420af6bfbade511 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:10.863814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3358f1-d3e4-46c3-8eff-35d9b3493cc9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.864100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3358f1-d3e4-46c3-8eff-35d9b3493cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.864377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3358f1-d3e4-46c3-8eff-35d9b3493cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.884635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ca3358f1-d3e4-46c3-8eff-35d9b3493cc9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:10.885791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca3358f1-d3e4-46c3-8eff-35d9b3493cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:10.939865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61847b-044c-439e-abdb-4848b3d2bcb4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:10.951065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 540/1074] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:10.951855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3358f1-d3e4-46c3-8eff-35d9b3493cc9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:10.976477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:10.977086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:10.977367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:10.977671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:10.978406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:10.978714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:10.980135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.000996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 535/1075] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:57:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:11.007404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.014038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:11.066763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3358f1-d3e4-46c3-8eff-35d9b3493cc9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.082070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:11.103317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.103979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.104260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.104543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511/users/f53672a05a434d3cb5b034820cd90ecc/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.106498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7636c6be385f4ea08420af6bfbade511, user_id=f53672a05a434d3cb5b034820cd90ecc, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.107932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.121690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542500ad-0ee0-4ff6-829c-bb0ef0cf58d4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.156325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.156892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.157173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.157447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.167656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.169291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.178755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 541/1076] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:57:10 2022] PUT /identity/v3/projects/7636c6be385f4ea08420af6bfbade511/users/f53672a05a434d3cb5b034820cd90ecc/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:11.185396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:11.186675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:57:11.187849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:11.187849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:11.187849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:11.190174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 536/1077] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:11.196997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.230745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.298110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:11.311729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f28c8-4626-44fc-bf3e-dbf27c7a25b9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.354359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a484a-b735-40f9-8147-da6165361ed9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.369854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.370511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.370791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.371064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.371714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:11.372017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.373385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.385367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 542/1078] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:11.391002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.391309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.391611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.391891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.392161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.428123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:11.467240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.468684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.497367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982052c-3171-436d-9316-db0686e8d218 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.529974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:11.533945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.534484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.534766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.535036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.542550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.543994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:57:11.556742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:11.557730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:11.563528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 543/1079] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:57:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:11.564026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.586207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 537/1080] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:57:11.591664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.669267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:11.677605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a5047-f7b2-486b-89b5-e4f1e1ef4ba7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.697322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de220b55-4b7a-4b3c-8ef4-e40ecab33248 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.714042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.714750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.715021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.718052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.718052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:11.718052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.718052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.731616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 544/1081] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:57:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:11.734355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.734992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.735275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.735547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.737487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.747040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.752727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:11.763856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:11.764948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:57:11.766079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:11.766079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:11.766079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:11.766517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 538/1082] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:11.772893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:11.782697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:11.860336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f8af6-f176-4d9b-88ee-9a5c3d872e21 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.875766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e3659-2371-42df-9407-4fe567f131b1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:11.897343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff31a21-0ae5-415d-9333-ba9a2407bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.898060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff31a21-0ae5-415d-9333-ba9a2407bf27 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.898344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff31a21-0ae5-415d-9333-ba9a2407bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.898613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff31a21-0ae5-415d-9333-ba9a2407bf27 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/46fd540df66143b2b768cc8d9f4beed7/roles/4dc70a805edd4526856e1731a0cbbcaa` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.900884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff31a21-0ae5-415d-9333-ba9a2407bf27 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=46fd540df66143b2b768cc8d9f4beed7, role_id=4dc70a805edd4526856e1731a0cbbcaa)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.902380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff31a21-0ae5-415d-9333-ba9a2407bf27 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.911074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1f56b-cd02-4fad-b4ff-d2cc8c3dce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:11.911808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1f56b-cd02-4fad-b4ff-d2cc8c3dce29 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:11.912147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1f56b-cd02-4fad-b4ff-d2cc8c3dce29 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:11.912486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1f56b-cd02-4fad-b4ff-d2cc8c3dce29 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:11.913188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1f56b-cd02-4fad-b4ff-d2cc8c3dce29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:11.913557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1f56b-cd02-4fad-b4ff-d2cc8c3dce29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:11.915000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1f56b-cd02-4fad-b4ff-d2cc8c3dce29 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:11.926674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 539/1083] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:12.003207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 545/1084] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:57:11 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/46fd540df66143b2b768cc8d9f4beed7/roles/4dc70a805edd4526856e1731a0cbbcaa => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 15:57:13.139940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e881a-cdd6-4745-8d5b-7b0e7e7e3126 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:13.140498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e881a-cdd6-4745-8d5b-7b0e7e7e3126 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:13.140498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e881a-cdd6-4745-8d5b-7b0e7e7e3126 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:13.141865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 540/1085] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:13.147677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fe1c-e938-4ca6-80aa-37174b9a00f4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:13.148075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fe1c-e938-4ca6-80aa-37174b9a00f4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:13.148233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fe1c-e938-4ca6-80aa-37174b9a00f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:13.166310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c088fe1c-e938-4ca6-80aa-37174b9a00f4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:13.167502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c088fe1c-e938-4ca6-80aa-37174b9a00f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:13.176503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:13.233286 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 546/1086] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:13.504997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e881a-cdd6-4745-8d5b-7b0e7e7e3126 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:13.569701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:13.605728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e881a-cdd6-4745-8d5b-7b0e7e7e3126 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:13.639154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:13.639779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:13.640064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:13.640341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:13.651251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:13.656077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:13.657196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:13.658096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 541/1087] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:13.663234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fe1c-e938-4ca6-80aa-37174b9a00f4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:13.763970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fe1c-e938-4ca6-80aa-37174b9a00f4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:13.796887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4716b-9840-4c18-a4c2-def7d480382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:13.797489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4716b-9840-4c18-a4c2-def7d480382c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:13.797793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4716b-9840-4c18-a4c2-def7d480382c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:13.798064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4716b-9840-4c18-a4c2-def7d480382c None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:13.798717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4716b-9840-4c18-a4c2-def7d480382c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:13.799017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4716b-9840-4c18-a4c2-def7d480382c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:13.800375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4716b-9840-4c18-a4c2-def7d480382c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:13.808793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 547/1088] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:13.814548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:13.911734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13568151-5e7c-4c8b-abb7-b7e8a05c40f3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:13.926860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580a68-4668-4f98-809d-e22020b23894 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:13.927694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580a68-4668-4f98-809d-e22020b23894 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:13.928077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580a68-4668-4f98-809d-e22020b23894 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:13.929429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 548/1089] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:13.934338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cad0-d44a-4c25-9a2e-385580a57f07 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:13.934625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cad0-d44a-4c25-9a2e-385580a57f07 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:13.934904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cad0-d44a-4c25-9a2e-385580a57f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:13.936771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:13.946016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:13.946657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:13.946941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:13.947216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] PATH_INFO: `/v3/users/f53672a05a434d3cb5b034820cd90ecc` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:13.953645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:13.953774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] RBAC: Authorizing `identity:get_user(user_id=f53672a05a434d3cb5b034820cd90ecc)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:13.953774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:13.953774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 542/1090] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:13 2022] GET /identity/v3/users/f53672a05a434d3cb5b034820cd90ecc => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:13.953967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3362cad0-d44a-4c25-9a2e-385580a57f07 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:13.955173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3362cad0-d44a-4c25-9a2e-385580a57f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:13.957288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:14.025366 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 549/1091] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:57:14.056470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce6e17-27ee-41cb-8023-0aa71cb00e72 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:14.088096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:14.088690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:14.088969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:14.089236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:14.151988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:14.159410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:14.160839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:14.262427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 543/1092] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 15:57:14.263826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362cad0-d44a-4c25-9a2e-385580a57f07 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:14.301351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:14.375258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362cad0-d44a-4c25-9a2e-385580a57f07 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:14.399075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:14.405546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308ea3-7c32-4143-a1af-e841755661e5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:14.410806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:14.412010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:14.412301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:14.412579 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:14.413741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] RBAC: Authorizing `identity:get_project(project_id=7636c6be385f4ea08420af6bfbade511)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:14.415182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:14.416873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 550/1093] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:14 2022] GET /identity/v3/projects/7636c6be385f4ea08420af6bfbade511 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:14.422886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:14.426291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:14.440155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:14.440787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:14.441068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:14.441340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:14.451622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:14.456530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:14.457480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:14.459095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 544/1094] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Sep 4 15:57:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:14.464198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:14.522943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ae5f0-7cae-42a3-bf24-cf80a6c847d5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:14.556284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:14.556821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:14.557113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:14.557365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:14.558142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=7636c6be385f4ea08420af6bfbade511, user.id=f53672a05a434d3cb5b034820cd90ecc) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:14.558454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:14.559942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:14.563323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5d85b-735e-4362-b45d-16c6384e9ac5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:14.565461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 551/1095] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:14 2022] GET /identity/v3/role_assignments?user.id=f53672a05a434d3cb5b034820cd90ecc&scope.project.id=7636c6be385f4ea08420af6bfbade511&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:14.602089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:14.602715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:14.602996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:14.603268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:14.603946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:14.604250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:14.605647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:14.620134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 545/1096] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Sep 4 15:57:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:57:14.621468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:14.692362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:14.727455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31741af-a159-47ee-9a91-731eae8b385f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:14.760839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:14.761464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:14.761767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:14.762037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:14.773004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:14.774463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:14.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:14.790036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:57:14.790969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:14.790969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:14.790969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:14.790969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 552/1097] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:14.795843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:14.814406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:14.895470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41336ef3-ab61-48f2-9cdd-6150e984e580 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:14.928889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:14.929492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:14.929803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:14.930077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:14.930715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:14.931016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:14.932368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:14.943943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 546/1098] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:14.949550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:15.048390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14286d6e-4b00-4cf9-a0f0-8b70302f2dfb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:15.068777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:15.084094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:15.084897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:15.085185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:15.085463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:15.092888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:15.094351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:57:15.106502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:15.107672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:15.108663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 553/1099] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Sep 4 15:57:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:15.113403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:15.189553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:15.212452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a9c47-2457-46fb-9659-3f735a47a810 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:15.246350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:15.246956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:15.247237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:15.247510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:15.248235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:15.248541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:15.250000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:15.267104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 547/1100] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Sep 4 15:57:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:15.273412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:15.373930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07ceb1-9f89-48b6-b125-e0d0bbfbc4c8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:15.406576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:15.407173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:15.407449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:15.407720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:15.454150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:15.474786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:15.476513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:15.574818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 554/1101] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 15:57:15.580768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:15.681666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b428-6671-4368-8bf6-ecb7922bb50e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:15.716153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:15.716779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:15.717057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:15.717329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:15.719191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:15.728029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:15.729449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:15.744055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:15.745179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:57:15.746113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:15.746113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:57:15.746113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:15.746113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 548/1102] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:15.750874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:15.828611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:15.847719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19cd2d-bef0-454e-b06c-a6f09d00f9d7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:15.883395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:15.883997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:15.884277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:15.884550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:15.885460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:15.885794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:15.887162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:15.899087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 555/1103] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:15.904718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:16.002372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3d4b4-8519-42de-bef9-399e4edf2eee None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:16.035676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:16.036308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:16.036689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:16.036963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:16.043504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:16.044934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:57:16.056741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:16.057680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:16.058585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 549/1104] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:57:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:16.063701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:16.163850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d40c32-2099-4425-85e9-98b54c3abc5b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:16.197444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94f8a2-5e4a-4e78-a911-4dc57c89cc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:16.198161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94f8a2-5e4a-4e78-a911-4dc57c89cc53 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:16.198446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94f8a2-5e4a-4e78-a911-4dc57c89cc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:16.198723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94f8a2-5e4a-4e78-a911-4dc57c89cc53 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:16.199410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94f8a2-5e4a-4e78-a911-4dc57c89cc53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:16.199716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94f8a2-5e4a-4e78-a911-4dc57c89cc53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:16.201140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94f8a2-5e4a-4e78-a911-4dc57c89cc53 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:16.210080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:16.213387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 556/1105] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:57:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:16.219477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:16.265945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:16.318550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd13e4-e168-439f-9e67-38d09354c32c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:16.353442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c82f8d-0b34-4c3d-8316-d072302246fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:16.354104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c82f8d-0b34-4c3d-8316-d072302246fd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:16.354380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c82f8d-0b34-4c3d-8316-d072302246fd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:16.354649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c82f8d-0b34-4c3d-8316-d072302246fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:16.355450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c82f8d-0b34-4c3d-8316-d072302246fd None admin] RBAC: Adding query filter params (role.id=4dc70a805edd4526856e1731a0cbbcaa, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=46fd540df66143b2b768cc8d9f4beed7) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:16.355675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c82f8d-0b34-4c3d-8316-d072302246fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:16.357089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c82f8d-0b34-4c3d-8316-d072302246fd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:16.362876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 550/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:16 2022] GET /identity/v3/role_assignments?user.id=46fd540df66143b2b768cc8d9f4beed7&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=4dc70a805edd4526856e1731a0cbbcaa => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:16.528923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd6c00-9b8b-43c7-a8dd-8c35f26761c3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:16.529412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd6c00-9b8b-43c7-a8dd-8c35f26761c3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:16.529412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd6c00-9b8b-43c7-a8dd-8c35f26761c3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:16.530793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 557/1107] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:16.536199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b820be9-3de3-4272-97fd-29f47b70a58b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:16.536804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b820be9-3de3-4272-97fd-29f47b70a58b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:16.537081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b820be9-3de3-4272-97fd-29f47b70a58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:16.555513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0b820be9-3de3-4272-97fd-29f47b70a58b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:16.556701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0b820be9-3de3-4272-97fd-29f47b70a58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:16.627061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 551/1108] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:16.901978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd6c00-9b8b-43c7-a8dd-8c35f26761c3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:17.003811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd6c00-9b8b-43c7-a8dd-8c35f26761c3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:17.040106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879583de-fc75-4d3d-a484-8287add49a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:17.040747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879583de-fc75-4d3d-a484-8287add49a44 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:17.041030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879583de-fc75-4d3d-a484-8287add49a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:17.041302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879583de-fc75-4d3d-a484-8287add49a44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:17.042531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879583de-fc75-4d3d-a484-8287add49a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:17.043969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879583de-fc75-4d3d-a484-8287add49a44 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:17.045556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 558/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:17.051534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b820be9-3de3-4272-97fd-29f47b70a58b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:17.104246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:17.162699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b820be9-3de3-4272-97fd-29f47b70a58b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:17.197866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e426c-8cdb-474e-8b53-fbb3b963a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:17.198475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e426c-8cdb-474e-8b53-fbb3b963a7a5 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:17.198752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e426c-8cdb-474e-8b53-fbb3b963a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:17.199023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e426c-8cdb-474e-8b53-fbb3b963a7a5 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:17.199680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e426c-8cdb-474e-8b53-fbb3b963a7a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:17.201095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e426c-8cdb-474e-8b53-fbb3b963a7a5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:17.292203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 552/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:57:17 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:19.280611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0fb4d-f34e-4686-b8e5-da9df65b6a5e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:19.281264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0fb4d-f34e-4686-b8e5-da9df65b6a5e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:19.281264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0fb4d-f34e-4686-b8e5-da9df65b6a5e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:19.282697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 559/1111] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:19.289707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d89b2-c8eb-49e3-b4bc-e23b26162797 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:19.290247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d89b2-c8eb-49e3-b4bc-e23b26162797 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:19.290557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d89b2-c8eb-49e3-b4bc-e23b26162797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:19.309913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-903d89b2-c8eb-49e3-b4bc-e23b26162797 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:19.311084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-903d89b2-c8eb-49e3-b4bc-e23b26162797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:19.378661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 553/1112] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:19.653983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0fb4d-f34e-4686-b8e5-da9df65b6a5e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:19.719679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:19.752115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0fb4d-f34e-4686-b8e5-da9df65b6a5e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:19.784921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:19.785536 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:19.785848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:19.786119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] PATH_INFO: `/v3/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:19.787565 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] RBAC: Authorizing `identity:get_role(role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:19.788923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:19.790259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 560/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:19 2022] GET /identity/v3/roles/eb0cba906bfe429883619da6e985a2b4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:19.796170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d89b2-c8eb-49e3-b4bc-e23b26162797 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:19.823989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:19.895212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d89b2-c8eb-49e3-b4bc-e23b26162797 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:19.929073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:19.929663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:19.929945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:19.930215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] PATH_INFO: `/v3/users/cbf3a27355e747b289cc8d6c5121a14f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:19.950917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] RBAC: Authorizing `identity:get_user(user_id=cbf3a27355e747b289cc8d6c5121a14f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:19.954842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:19.960218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 554/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:19 2022] GET /identity/v3/users/cbf3a27355e747b289cc8d6c5121a14f => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:19.962591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:20.062865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5a300-f1e5-44cd-994c-242e25611ced None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:20.096165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:20.096764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:20.097043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:20.097317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:20.101654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:20.165068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:20.166559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:20.268131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 561/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:57:20.274953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:20.358639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:20.374070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f811d9e-ba27-43aa-9cf7-b01e5664c984 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:20.408210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b515f-351a-4a05-a4a7-462c46208a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:20.408822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b515f-351a-4a05-a4a7-462c46208a66 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:20.409103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b515f-351a-4a05-a4a7-462c46208a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:20.409373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b515f-351a-4a05-a4a7-462c46208a66 None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:20.410497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b515f-351a-4a05-a4a7-462c46208a66 None admin] RBAC: Authorizing `identity:get_project(project_id=7636c6be385f4ea08420af6bfbade511)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:20.412357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b515f-351a-4a05-a4a7-462c46208a66 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:20.413992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 555/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:20 2022] GET /identity/v3/projects/7636c6be385f4ea08420af6bfbade511 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:20.420141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:20.454793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:20.518443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379540c-bbae-4e59-89e6-1d1e7d4c36a4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:20.550838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58ad3d-0e78-4204-8d5e-8ae8c0265077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:20.551456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58ad3d-0e78-4204-8d5e-8ae8c0265077 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:20.551723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58ad3d-0e78-4204-8d5e-8ae8c0265077 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:20.552725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58ad3d-0e78-4204-8d5e-8ae8c0265077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:20.553840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58ad3d-0e78-4204-8d5e-8ae8c0265077 None admin] RBAC: Adding query filter params (role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=7636c6be385f4ea08420af6bfbade511, user.id=cbf3a27355e747b289cc8d6c5121a14f) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:20.554355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58ad3d-0e78-4204-8d5e-8ae8c0265077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:20.556620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58ad3d-0e78-4204-8d5e-8ae8c0265077 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:20.564786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 562/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:20 2022] GET /identity/v3/role_assignments?user.id=cbf3a27355e747b289cc8d6c5121a14f&scope.project.id=7636c6be385f4ea08420af6bfbade511&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:22.431594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42fe0-577d-4072-9a24-f1028f1ef4c8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:22.432118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42fe0-577d-4072-9a24-f1028f1ef4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:22.432118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42fe0-577d-4072-9a24-f1028f1ef4c8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:22.433456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 556/1118] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:22.438448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664191dd-14f6-48ba-b5d6-7b552bd8ca65 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:22.438890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664191dd-14f6-48ba-b5d6-7b552bd8ca65 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:22.438987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664191dd-14f6-48ba-b5d6-7b552bd8ca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:22.457237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-664191dd-14f6-48ba-b5d6-7b552bd8ca65 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:22.459203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-664191dd-14f6-48ba-b5d6-7b552bd8ca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:22.523582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 563/1119] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:22.793161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de42fe0-577d-4072-9a24-f1028f1ef4c8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:22.891554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de42fe0-577d-4072-9a24-f1028f1ef4c8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:22.923672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:22.924299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:22.924573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:22.924844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] PATH_INFO: `/v3/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:22.926249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] RBAC: Authorizing `identity:get_role(role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:22.928118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:22.929396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 557/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:22 2022] GET /identity/v3/roles/eb0cba906bfe429883619da6e985a2b4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:22.935125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664191dd-14f6-48ba-b5d6-7b552bd8ca65 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:22.964378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:23.031640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664191dd-14f6-48ba-b5d6-7b552bd8ca65 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:23.064907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:23.065473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:23.065797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:23.066075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] PATH_INFO: `/v3/users/cbf3a27355e747b289cc8d6c5121a14f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:23.067198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] RBAC: Authorizing `identity:get_user(user_id=cbf3a27355e747b289cc8d6c5121a14f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:23.068666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:23.070268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 564/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:22 2022] GET /identity/v3/users/cbf3a27355e747b289cc8d6c5121a14f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:23.076076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:23.100657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:23.178348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412727fa-3c6f-4db0-adc3-4c34d11fbf4f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:23.210930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bf80b-252b-4c68-aa32-7966c68a2936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:23.211549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bf80b-252b-4c68-aa32-7966c68a2936 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:23.211827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bf80b-252b-4c68-aa32-7966c68a2936 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:23.212096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bf80b-252b-4c68-aa32-7966c68a2936 None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:23.213195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bf80b-252b-4c68-aa32-7966c68a2936 None admin] RBAC: Authorizing `identity:get_project(project_id=7636c6be385f4ea08420af6bfbade511)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:23.214642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bf80b-252b-4c68-aa32-7966c68a2936 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:23.216279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 558/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:23 2022] GET /identity/v3/projects/7636c6be385f4ea08420af6bfbade511 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:23.222342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:23.314343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:23.317896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11926b-af37-4f8b-afc3-4c1cec47d7a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:23.351520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926ba5f-5f4b-4cf4-9538-5e6de229833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:23.352178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926ba5f-5f4b-4cf4-9538-5e6de229833b None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:23.352463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926ba5f-5f4b-4cf4-9538-5e6de229833b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:23.352744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926ba5f-5f4b-4cf4-9538-5e6de229833b None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511/users/cbf3a27355e747b289cc8d6c5121a14f/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:23.354917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926ba5f-5f4b-4cf4-9538-5e6de229833b None admin] RBAC: Authorizing `identity:create_grant(project_id=7636c6be385f4ea08420af6bfbade511, user_id=cbf3a27355e747b289cc8d6c5121a14f, role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:23.356362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926ba5f-5f4b-4cf4-9538-5e6de229833b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:23.451197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 565/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:57:23 2022] PUT /identity/v3/projects/7636c6be385f4ea08420af6bfbade511/users/cbf3a27355e747b289cc8d6c5121a14f/roles/eb0cba906bfe429883619da6e985a2b4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:57:25.338269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ef7b-2b97-4426-9913-e1be602d42d1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:25.338745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ef7b-2b97-4426-9913-e1be602d42d1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:25.338837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ef7b-2b97-4426-9913-e1be602d42d1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:25.340126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 559/1124] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:25.345279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad22056-9b48-4cd1-999b-4164133e4552 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:25.345725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad22056-9b48-4cd1-999b-4164133e4552 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:25.345849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad22056-9b48-4cd1-999b-4164133e4552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:25.363796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1ad22056-9b48-4cd1-999b-4164133e4552 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:25.365086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ad22056-9b48-4cd1-999b-4164133e4552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:25.428823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 566/1125] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:25.704247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ef7b-2b97-4426-9913-e1be602d42d1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:25.793717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:25.806170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ef7b-2b97-4426-9913-e1be602d42d1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:25.839666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:25.840281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:25.840561 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:25.840831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] PATH_INFO: `/v3/roles/eb0cba906bfe429883619da6e985a2b4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:25.842215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] RBAC: Authorizing `identity:get_role(role_id=eb0cba906bfe429883619da6e985a2b4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:25.843563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:25.844848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 560/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:25 2022] GET /identity/v3/roles/eb0cba906bfe429883619da6e985a2b4 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:25.851572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad22056-9b48-4cd1-999b-4164133e4552 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:25.935352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:25.947236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad22056-9b48-4cd1-999b-4164133e4552 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:25.980262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:25.980813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:25.981090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:25.981361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] PATH_INFO: `/v3/users/cbf3a27355e747b289cc8d6c5121a14f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:25.982482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] RBAC: Authorizing `identity:get_user(user_id=cbf3a27355e747b289cc8d6c5121a14f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:25.983908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:25.985520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 567/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:25 2022] GET /identity/v3/users/cbf3a27355e747b289cc8d6c5121a14f => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:25.992094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:26.089928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388294ea-e11b-48fa-b8c6-6acb4313871d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:26.123037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:26.123627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:26.123917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:26.124193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:26.148814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:26.192807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:26.194616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:26.290348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 561/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:57:26.296253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:26.394831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf48f3-54a0-4e98-b0ce-faa75124c0ce None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:26.428272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488b3f7-902c-44cf-8b1a-2d5fe3437776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:26.428895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b3f7-902c-44cf-8b1a-2d5fe3437776 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:26.429178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b3f7-902c-44cf-8b1a-2d5fe3437776 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:26.429454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b3f7-902c-44cf-8b1a-2d5fe3437776 None admin] PATH_INFO: `/v3/projects/7636c6be385f4ea08420af6bfbade511` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:26.430600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b3f7-902c-44cf-8b1a-2d5fe3437776 None admin] RBAC: Authorizing `identity:get_project(project_id=7636c6be385f4ea08420af6bfbade511)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:26.432038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b3f7-902c-44cf-8b1a-2d5fe3437776 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:26.433692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 568/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:26 2022] GET /identity/v3/projects/7636c6be385f4ea08420af6bfbade511 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:26.439771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:26.494127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:26.536191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f084ae-a853-4c0f-a387-dc4fdd1532e3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:26.569774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a67b9-e858-4360-93de-db8fbd97bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:26.570312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a67b9-e858-4360-93de-db8fbd97bbca None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:26.570590 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a67b9-e858-4360-93de-db8fbd97bbca None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:26.570863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a67b9-e858-4360-93de-db8fbd97bbca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:26.571577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a67b9-e858-4360-93de-db8fbd97bbca None admin] RBAC: Adding query filter params (role.id=eb0cba906bfe429883619da6e985a2b4, scope.project.id=7636c6be385f4ea08420af6bfbade511, user.id=cbf3a27355e747b289cc8d6c5121a14f) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:26.571881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a67b9-e858-4360-93de-db8fbd97bbca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:26.573293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a67b9-e858-4360-93de-db8fbd97bbca None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:26.578784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 562/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:26 2022] GET /identity/v3/role_assignments?user.id=cbf3a27355e747b289cc8d6c5121a14f&scope.project.id=7636c6be385f4ea08420af6bfbade511&role.id=eb0cba906bfe429883619da6e985a2b4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:28.483628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9505dbe-30e5-4628-891b-bcf597aa2a0b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:28.484125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9505dbe-30e5-4628-891b-bcf597aa2a0b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:28.484125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9505dbe-30e5-4628-891b-bcf597aa2a0b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:28.485446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 569/1131] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:28.490509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49078d0d-832b-4a87-8cf1-256447b3cd77 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:28.490879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49078d0d-832b-4a87-8cf1-256447b3cd77 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:28.491060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49078d0d-832b-4a87-8cf1-256447b3cd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:28.509519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-49078d0d-832b-4a87-8cf1-256447b3cd77 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:28.510711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-49078d0d-832b-4a87-8cf1-256447b3cd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:28.573791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 563/1132] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:28.840312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9505dbe-30e5-4628-891b-bcf597aa2a0b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:28.865691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:28.939841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9505dbe-30e5-4628-891b-bcf597aa2a0b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:28.977437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de30c0-b508-4885-9873-3117ec3153b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:28.978104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de30c0-b508-4885-9873-3117ec3153b9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:28.978387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de30c0-b508-4885-9873-3117ec3153b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:28.978662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de30c0-b508-4885-9873-3117ec3153b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:28.979781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de30c0-b508-4885-9873-3117ec3153b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:28.981220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de30c0-b508-4885-9873-3117ec3153b9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:28.982826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 570/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:28.988604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49078d0d-832b-4a87-8cf1-256447b3cd77 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:29.037151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:29.086066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49078d0d-832b-4a87-8cf1-256447b3cd77 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:29.118197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b48fe-e3c8-41cc-b3a9-81f878140176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:29.118714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b48fe-e3c8-41cc-b3a9-81f878140176 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:29.118989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b48fe-e3c8-41cc-b3a9-81f878140176 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:29.119259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b48fe-e3c8-41cc-b3a9-81f878140176 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:29.119892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b48fe-e3c8-41cc-b3a9-81f878140176 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:29.121452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b48fe-e3c8-41cc-b3a9-81f878140176 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:29.198911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 564/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:57:28 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:31.128768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d442b40-c0be-4989-a4ca-39179475e951 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:31.129302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d442b40-c0be-4989-a4ca-39179475e951 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:31.129302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d442b40-c0be-4989-a4ca-39179475e951 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:31.130972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 571/1135] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:31.136279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7c106-028d-433c-addc-c3026587fe56 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:31.136727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7c106-028d-433c-addc-c3026587fe56 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:31.136820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7c106-028d-433c-addc-c3026587fe56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:31.155550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a6b7c106-028d-433c-addc-c3026587fe56 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:31.156723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a6b7c106-028d-433c-addc-c3026587fe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:31.221707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 565/1136] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:31.492671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d442b40-c0be-4989-a4ca-39179475e951 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:31.580660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:31.590390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d442b40-c0be-4989-a4ca-39179475e951 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:31.621745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87071e6-0430-4e65-908a-681867d1e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:31.622363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87071e6-0430-4e65-908a-681867d1e479 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:31.622642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87071e6-0430-4e65-908a-681867d1e479 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:31.622914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87071e6-0430-4e65-908a-681867d1e479 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:31.624006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87071e6-0430-4e65-908a-681867d1e479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:31.625529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87071e6-0430-4e65-908a-681867d1e479 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:31.627757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 572/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:57:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:31.633823 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7c106-028d-433c-addc-c3026587fe56 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:31.666788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:31.734391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7c106-028d-433c-addc-c3026587fe56 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:31.768495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126ed0a-6962-4991-8d72-9b37524691ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:31.769031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126ed0a-6962-4991-8d72-9b37524691ed None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:31.769311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126ed0a-6962-4991-8d72-9b37524691ed None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:31.769607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126ed0a-6962-4991-8d72-9b37524691ed None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:31.770377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126ed0a-6962-4991-8d72-9b37524691ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:31.772164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126ed0a-6962-4991-8d72-9b37524691ed None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:31.905453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 566/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:57:31 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:33.875125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3908ba7-01e1-4357-ba24-4f1f8b0c53dc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:33.875650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3908ba7-01e1-4357-ba24-4f1f8b0c53dc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:33.875650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3908ba7-01e1-4357-ba24-4f1f8b0c53dc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:33.877270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 573/1139] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:33.882969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373c9c-0688-4a63-b6a7-0aec6e355d62 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:33.883356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373c9c-0688-4a63-b6a7-0aec6e355d62 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:33.883522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373c9c-0688-4a63-b6a7-0aec6e355d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:33.901785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-79373c9c-0688-4a63-b6a7-0aec6e355d62 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:33.902949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-79373c9c-0688-4a63-b6a7-0aec6e355d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:33.966656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 567/1140] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:34.240430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3908ba7-01e1-4357-ba24-4f1f8b0c53dc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:34.340928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3908ba7-01e1-4357-ba24-4f1f8b0c53dc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:34.373944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:34.374562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:34.374843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:34.375116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:34.385691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:34.390498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:34.391662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:34.392914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 574/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:57:34.398405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373c9c-0688-4a63-b6a7-0aec6e355d62 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:34.415829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:34.493961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373c9c-0688-4a63-b6a7-0aec6e355d62 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:34.526886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:34.527480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:34.527756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:34.528026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:34.528670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:34.528967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:34.530507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:34.538248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 568/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:34.544059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:34.568628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:34.643857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ba4d-3844-4592-82e1-ff881a385836 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:34.677105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:34.677746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:34.678030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:34.678300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] PATH_INFO: `/v3/users/697ef9a3f1c84f9eab6158f06214c6b9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:34.691455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] RBAC: Authorizing `identity:get_user(user_id=697ef9a3f1c84f9eab6158f06214c6b9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:34.693601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:34.695344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 575/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:34 2022] GET /identity/v3/users/697ef9a3f1c84f9eab6158f06214c6b9 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:34.701276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:34.779314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:34.797268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf311c-03f0-4810-a439-fe9a2bcb0f67 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:34.831936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:34.832529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:34.832859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:34.833131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:34.899125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:34.900564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:34.978272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:34.997349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 569/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:57:35.003574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:35.079819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:35.100401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d4a4-5e19-45f0-a8cd-d9aed24d5b39 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:35.133543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ada87-a967-4c95-bb5e-8ac8ba0c292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:35.134185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ada87-a967-4c95-bb5e-8ac8ba0c292f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:35.134461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ada87-a967-4c95-bb5e-8ac8ba0c292f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:35.134729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ada87-a967-4c95-bb5e-8ac8ba0c292f None admin] PATH_INFO: `/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:35.135852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ada87-a967-4c95-bb5e-8ac8ba0c292f None admin] RBAC: Authorizing `identity:get_project(project_id=e2f22488d81f4d8eb2968e7a14d5b297)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:35.137272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ada87-a967-4c95-bb5e-8ac8ba0c292f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:35.138887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 576/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:35 2022] GET /identity/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:35.145254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:35.248327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f18a83-ee0e-4b8b-a921-4c73bf875eb2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:35.280714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4156f-faa6-4a3d-9dce-5319ffc4e8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:35.281304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4156f-faa6-4a3d-9dce-5319ffc4e8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:35.281601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4156f-faa6-4a3d-9dce-5319ffc4e8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:35.281875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4156f-faa6-4a3d-9dce-5319ffc4e8d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:35.282589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4156f-faa6-4a3d-9dce-5319ffc4e8d1 None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=e2f22488d81f4d8eb2968e7a14d5b297, user.id=697ef9a3f1c84f9eab6158f06214c6b9) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:35.282891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4156f-faa6-4a3d-9dce-5319ffc4e8d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:35.284299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4156f-faa6-4a3d-9dce-5319ffc4e8d1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:35.290026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 570/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:35 2022] GET /identity/v3/role_assignments?user.id=697ef9a3f1c84f9eab6158f06214c6b9&scope.project.id=e2f22488d81f4d8eb2968e7a14d5b297&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:37.165516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc5463-a4fe-42b7-9a81-d35b37075cc4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:37.166096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc5463-a4fe-42b7-9a81-d35b37075cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:37.166096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc5463-a4fe-42b7-9a81-d35b37075cc4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:37.167446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 577/1147] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:37.173360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60173840-57c0-4bb2-b712-4d48a3caec1d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:37.173819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60173840-57c0-4bb2-b712-4d48a3caec1d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:37.174098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60173840-57c0-4bb2-b712-4d48a3caec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:37.193885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-60173840-57c0-4bb2-b712-4d48a3caec1d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:37.195075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-60173840-57c0-4bb2-b712-4d48a3caec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:37.234214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:37.261355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 571/1148] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:37.536186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc5463-a4fe-42b7-9a81-d35b37075cc4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:37.634475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc5463-a4fe-42b7-9a81-d35b37075cc4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:37.667225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:37.667842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:37.668124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:37.668400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:37.678589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:37.683460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:37.684550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:37.685766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 578/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:37.690502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60173840-57c0-4bb2-b712-4d48a3caec1d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:37.787211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60173840-57c0-4bb2-b712-4d48a3caec1d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:37.820411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:37.821012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:37.821294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:37.821596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:37.822260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:37.822566 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:37.823922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:37.831680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 572/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:37.837406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:37.846481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:37.935817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f8339-437d-4302-8968-04442906b6c2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:37.968675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:37.969298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:37.969604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:37.969888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] PATH_INFO: `/v3/users/697ef9a3f1c84f9eab6158f06214c6b9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:37.971008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] RBAC: Authorizing `identity:get_user(user_id=697ef9a3f1c84f9eab6158f06214c6b9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:37.972442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:37.974135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 579/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:37 2022] GET /identity/v3/users/697ef9a3f1c84f9eab6158f06214c6b9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:37.980010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:38.015364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:38.077414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685e59e-2f86-4454-8571-37d98c3f6741 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:38.110801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eedea-8229-4023-a8e8-d036a285d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:38.111415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eedea-8229-4023-a8e8-d036a285d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:38.111690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eedea-8229-4023-a8e8-d036a285d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:38.111960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eedea-8229-4023-a8e8-d036a285d8c8 None admin] PATH_INFO: `/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:38.113051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eedea-8229-4023-a8e8-d036a285d8c8 None admin] RBAC: Authorizing `identity:get_project(project_id=e2f22488d81f4d8eb2968e7a14d5b297)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:38.114546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eedea-8229-4023-a8e8-d036a285d8c8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:38.116110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 573/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:37 2022] GET /identity/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:38.122055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:38.194428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:38.218189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c14594-c364-4d23-97e4-c8d5cfa1b931 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:38.257033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2d2f4-16ac-4841-ba86-c301e7751c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:38.257724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2d2f4-16ac-4841-ba86-c301e7751c12 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:38.258008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2d2f4-16ac-4841-ba86-c301e7751c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:38.258283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2d2f4-16ac-4841-ba86-c301e7751c12 None admin] PATH_INFO: `/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297/users/697ef9a3f1c84f9eab6158f06214c6b9/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:38.260166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2d2f4-16ac-4841-ba86-c301e7751c12 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2f22488d81f4d8eb2968e7a14d5b297, user_id=697ef9a3f1c84f9eab6158f06214c6b9, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:38.261629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2d2f4-16ac-4841-ba86-c301e7751c12 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:38.362531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 580/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:57:38 2022] PUT /identity/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297/users/697ef9a3f1c84f9eab6158f06214c6b9/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 15:57:40.267857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38416e-d38b-4081-8003-c8103fcd7546 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:40.268389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38416e-d38b-4081-8003-c8103fcd7546 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:40.268506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38416e-d38b-4081-8003-c8103fcd7546 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:40.269768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 574/1154] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:40.275857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8b8be-7bfb-4f4f-adcd-c4ce76552836 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:40.276229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8b8be-7bfb-4f4f-adcd-c4ce76552836 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:40.276413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8b8be-7bfb-4f4f-adcd-c4ce76552836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:40.295064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9cd8b8be-7bfb-4f4f-adcd-c4ce76552836 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:40.296250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9cd8b8be-7bfb-4f4f-adcd-c4ce76552836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:40.362195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 581/1155] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:40.639510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38416e-d38b-4081-8003-c8103fcd7546 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:40.738523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38416e-d38b-4081-8003-c8103fcd7546 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:40.741400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:40.772578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:40.773194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:40.773471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:40.773791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:40.783679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:40.789095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:40.790248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:40.791338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 575/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:40.796184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8b8be-7bfb-4f4f-adcd-c4ce76552836 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:40.857302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:40.893452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8b8be-7bfb-4f4f-adcd-c4ce76552836 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:40.930469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:40.931072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:40.931353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:40.931627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:40.932287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:40.932591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:40.934061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:40.941832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 582/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:40.947761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:41.046531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c203ee8-e5e1-46e6-a47a-071fefbdda0a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:41.079367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:41.079986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:41.080270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:41.080547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] PATH_INFO: `/v3/users/697ef9a3f1c84f9eab6158f06214c6b9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:41.081696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] RBAC: Authorizing `identity:get_user(user_id=697ef9a3f1c84f9eab6158f06214c6b9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:41.083287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:41.084827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 576/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:40 2022] GET /identity/v3/users/697ef9a3f1c84f9eab6158f06214c6b9 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:41.090776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:41.194228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee4ca4-9fef-421f-a5d1-c73f38b1b2be None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:41.219169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:41.227745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:41.228348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:41.233235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:41.233235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:41.294245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:41.295713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:41.389468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 583/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:57:41.396040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:41.427343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:41.493222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb7242-4779-411a-90d1-397d945d2013 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:41.526175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2784bec-3b16-4d6a-a92a-36ce6b3306bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:41.526792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784bec-3b16-4d6a-a92a-36ce6b3306bb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:41.527066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784bec-3b16-4d6a-a92a-36ce6b3306bb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:41.527331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784bec-3b16-4d6a-a92a-36ce6b3306bb None admin] PATH_INFO: `/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:41.528410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2784bec-3b16-4d6a-a92a-36ce6b3306bb None admin] RBAC: Authorizing `identity:get_project(project_id=e2f22488d81f4d8eb2968e7a14d5b297)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:41.529842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2784bec-3b16-4d6a-a92a-36ce6b3306bb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:41.531427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 577/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:41 2022] GET /identity/v3/projects/e2f22488d81f4d8eb2968e7a14d5b297 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:41.538073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:41.564516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:41.635313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c767f86-61bf-4dae-bd07-e5f237fbb9ae None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:41.667962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cfc43-82aa-455c-96b7-04128e68a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:41.668559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cfc43-82aa-455c-96b7-04128e68a89e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:41.668844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cfc43-82aa-455c-96b7-04128e68a89e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:41.669124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cfc43-82aa-455c-96b7-04128e68a89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:41.669872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cfc43-82aa-455c-96b7-04128e68a89e None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=e2f22488d81f4d8eb2968e7a14d5b297, user.id=697ef9a3f1c84f9eab6158f06214c6b9) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:41.670184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cfc43-82aa-455c-96b7-04128e68a89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:41.671605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cfc43-82aa-455c-96b7-04128e68a89e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:41.677876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 584/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:41 2022] GET /identity/v3/role_assignments?user.id=697ef9a3f1c84f9eab6158f06214c6b9&scope.project.id=e2f22488d81f4d8eb2968e7a14d5b297&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:43.605652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8436-bc1b-4faa-99dd-a716a5934373 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:43.606158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8436-bc1b-4faa-99dd-a716a5934373 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:43.606158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8436-bc1b-4faa-99dd-a716a5934373 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:43.607434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 578/1162] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:43.612499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e355ea1-a94c-40d6-9e5d-1b0882abfc08 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:43.612875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e355ea1-a94c-40d6-9e5d-1b0882abfc08 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:43.613066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e355ea1-a94c-40d6-9e5d-1b0882abfc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:43.631806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7e355ea1-a94c-40d6-9e5d-1b0882abfc08 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:43.632995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7e355ea1-a94c-40d6-9e5d-1b0882abfc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:43.698525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 585/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:43.967700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8436-bc1b-4faa-99dd-a716a5934373 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:44.062342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:44.066108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8436-bc1b-4faa-99dd-a716a5934373 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:44.098863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:44.099447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:44.099722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:44.099987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:44.164108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:44.165532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:44.258746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:44.263665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 579/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:57:44.269741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e355ea1-a94c-40d6-9e5d-1b0882abfc08 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:44.291267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:44.368950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e355ea1-a94c-40d6-9e5d-1b0882abfc08 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:44.403055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe6dcf-06c6-412b-aa85-1d74f0fabfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:44.403677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6dcf-06c6-412b-aa85-1d74f0fabfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:44.403958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6dcf-06c6-412b-aa85-1d74f0fabfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:44.404230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6dcf-06c6-412b-aa85-1d74f0fabfe5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:44.411215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe6dcf-06c6-412b-aa85-1d74f0fabfe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:44.412645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe6dcf-06c6-412b-aa85-1d74f0fabfe5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4afe6dcf-06c6-412b-aa85-1d74f0fabfe5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:44.423910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:44.424921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:57:44.425871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 586/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:57:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:44.431045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:44.528428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99607057-ce71-41c3-97fa-c38e987f7f5c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:44.568081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70177d03-2708-431c-aa5c-48e138df6525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:44.568684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70177d03-2708-431c-aa5c-48e138df6525 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:44.568970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70177d03-2708-431c-aa5c-48e138df6525 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:44.569239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70177d03-2708-431c-aa5c-48e138df6525 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:44.569907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70177d03-2708-431c-aa5c-48e138df6525 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:44.570208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70177d03-2708-431c-aa5c-48e138df6525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:44.571555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70177d03-2708-431c-aa5c-48e138df6525 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:44.578534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 580/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Sep 4 15:57:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:46.467313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cad6ae-b8c1-4eae-8e14-439fda0ce9b8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:46.467816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cad6ae-b8c1-4eae-8e14-439fda0ce9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:46.467905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cad6ae-b8c1-4eae-8e14-439fda0ce9b8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:46.469244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 587/1167] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:46.474474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f1e9-f4a6-46db-821c-39dc1302d684 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:46.474859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f1e9-f4a6-46db-821c-39dc1302d684 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:46.475029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f1e9-f4a6-46db-821c-39dc1302d684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:46.492705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3615f1e9-f4a6-46db-821c-39dc1302d684 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:46.493911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3615f1e9-f4a6-46db-821c-39dc1302d684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:46.551660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:46.559614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 581/1168] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:46.829238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cad6ae-b8c1-4eae-8e14-439fda0ce9b8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:46.908730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:46.932611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cad6ae-b8c1-4eae-8e14-439fda0ce9b8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:46.966055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1daae-d147-4916-b694-e9795273f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:46.966654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1daae-d147-4916-b694-e9795273f7d6 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:46.966937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1daae-d147-4916-b694-e9795273f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:46.967216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1daae-d147-4916-b694-e9795273f7d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:46.967893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1daae-d147-4916-b694-e9795273f7d6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:46.969260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1daae-d147-4916-b694-e9795273f7d6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:47.038322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 588/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Sun Sep 4 15:57:46 2022] POST /identity/v3/domains => generated 267 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:48.969851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cd15f-2647-4842-b592-0dc41bc3689c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:48.970408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cd15f-2647-4842-b592-0dc41bc3689c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:48.970408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cd15f-2647-4842-b592-0dc41bc3689c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:48.971727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 582/1170] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:48.977321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502a9d-59cf-45c0-8efd-db712f58712f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:48.977782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502a9d-59cf-45c0-8efd-db712f58712f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:48.977917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc502a9d-59cf-45c0-8efd-db712f58712f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:48.996236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cc502a9d-59cf-45c0-8efd-db712f58712f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:48.997416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cc502a9d-59cf-45c0-8efd-db712f58712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:49.063458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 589/1171] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:49.330638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cd15f-2647-4842-b592-0dc41bc3689c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:49.431440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cd15f-2647-4842-b592-0dc41bc3689c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:49.464625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b87e9f-21b4-4cce-bd13-75d17742319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:49.465244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b87e9f-21b4-4cce-bd13-75d17742319d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:49.465530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b87e9f-21b4-4cce-bd13-75d17742319d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:49.465826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b87e9f-21b4-4cce-bd13-75d17742319d None admin] PATH_INFO: `/v3/domains/a27623033b3142e590f3ab4525ddafd4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:49.476013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b87e9f-21b4-4cce-bd13-75d17742319d None admin] RBAC: Authorizing `identity:get_domain(domain_id=a27623033b3142e590f3ab4525ddafd4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:49.477432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b87e9f-21b4-4cce-bd13-75d17742319d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:49.479013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 583/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Sun Sep 4 15:57:49 2022] GET /identity/v3/domains/a27623033b3142e590f3ab4525ddafd4 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:49.484795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502a9d-59cf-45c0-8efd-db712f58712f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:49.551139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:49.579936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc502a9d-59cf-45c0-8efd-db712f58712f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:49.612553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11c0ea-db7d-4112-bd76-622e9c297970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:49.613151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11c0ea-db7d-4112-bd76-622e9c297970 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:49.613604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11c0ea-db7d-4112-bd76-622e9c297970 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:49.613847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11c0ea-db7d-4112-bd76-622e9c297970 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:49.614487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11c0ea-db7d-4112-bd76-622e9c297970 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:49.615910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11c0ea-db7d-4112-bd76-622e9c297970 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:49.734755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 590/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:57:49 2022] POST /identity/v3/projects => generated 361 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:51.713672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e4168-80b4-4de9-a834-61e86e5d016e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:51.714295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e4168-80b4-4de9-a834-61e86e5d016e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:51.714295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e4168-80b4-4de9-a834-61e86e5d016e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:51.715474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 584/1174] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:51.720770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483416-998d-4cf8-8ab3-0f7a2cffaff8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:51.721302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483416-998d-4cf8-8ab3-0f7a2cffaff8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:51.721419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483416-998d-4cf8-8ab3-0f7a2cffaff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:51.740399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3c483416-998d-4cf8-8ab3-0f7a2cffaff8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:51.741600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3c483416-998d-4cf8-8ab3-0f7a2cffaff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:51.806767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 591/1175] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:57:52.074395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e4168-80b4-4de9-a834-61e86e5d016e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:52.129793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:52.174471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e4168-80b4-4de9-a834-61e86e5d016e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:52.206024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7391b94-f8b5-459d-a6ba-33fe435824d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:52.206633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391b94-f8b5-459d-a6ba-33fe435824d1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:52.206910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391b94-f8b5-459d-a6ba-33fe435824d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:52.207698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391b94-f8b5-459d-a6ba-33fe435824d1 None admin] PATH_INFO: `/v3/domains/a27623033b3142e590f3ab4525ddafd4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:52.208847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7391b94-f8b5-459d-a6ba-33fe435824d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a27623033b3142e590f3ab4525ddafd4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:52.210288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7391b94-f8b5-459d-a6ba-33fe435824d1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:52.211814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 585/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Sun Sep 4 15:57:52 2022] GET /identity/v3/domains/a27623033b3142e590f3ab4525ddafd4 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:52.217486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c483416-998d-4cf8-8ab3-0f7a2cffaff8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:52.268048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:52.318263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c483416-998d-4cf8-8ab3-0f7a2cffaff8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:52.353089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930061f-1da7-4111-98f6-678b0f0a0dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:52.353732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930061f-1da7-4111-98f6-678b0f0a0dba None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:52.354017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930061f-1da7-4111-98f6-678b0f0a0dba None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:52.354291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930061f-1da7-4111-98f6-678b0f0a0dba None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:52.354939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930061f-1da7-4111-98f6-678b0f0a0dba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:52.356368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930061f-1da7-4111-98f6-678b0f0a0dba None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:52.450889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 592/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:57:52 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:57:54.403932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69892b95-3c2d-4598-92eb-9adeed4e6afb None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:54.404434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69892b95-3c2d-4598-92eb-9adeed4e6afb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:54.404532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69892b95-3c2d-4598-92eb-9adeed4e6afb None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:54.405848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 586/1178] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:54.411075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daf773-dff2-4b23-98cb-4643d623eacc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:54.411509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daf773-dff2-4b23-98cb-4643d623eacc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:54.411660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daf773-dff2-4b23-98cb-4643d623eacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:54.430920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-87daf773-dff2-4b23-98cb-4643d623eacc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:54.432098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-87daf773-dff2-4b23-98cb-4643d623eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:54.498468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 593/1179] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:54.769499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69892b95-3c2d-4598-92eb-9adeed4e6afb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:54.870260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69892b95-3c2d-4598-92eb-9adeed4e6afb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:54.892853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:54.904505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:54.905117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:54.905394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:54.905692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:54.915959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:54.921078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:54.922402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:54.923598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 587/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:54.927951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daf773-dff2-4b23-98cb-4643d623eacc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:54.968551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:55.034089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daf773-dff2-4b23-98cb-4643d623eacc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:55.067211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:55.067818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:55.068098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:55.068371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:55.069036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:55.069339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:55.070754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:55.078509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 594/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:55.085207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:55.230671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f18ac6-d26d-4c14-8384-13a21432bfc7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:55.267951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:55.268586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:55.268862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:55.269135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] PATH_INFO: `/v3/users/9439fe2941104fccaad35d200fff8ea0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:55.291824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] RBAC: Authorizing `identity:get_user(user_id=9439fe2941104fccaad35d200fff8ea0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:55.293024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:55.294728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 588/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:55 2022] GET /identity/v3/users/9439fe2941104fccaad35d200fff8ea0 => generated 311 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:55.300901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:55.419147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:55.422121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976979b-db4b-47e3-8072-2250e0ec45a1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:55.459296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:55.459973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:55.461650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:55.461964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:55.550522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:55.555100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:55.675602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:55.679735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 595/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:57:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:57:55.685627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:55.739780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:55.811876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b78fb4-b33a-48fe-9efb-97cfe11a4b9d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:55.863462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34e9c7-3be4-4a88-b4ce-fa3b1492f823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:55.864191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e9c7-3be4-4a88-b4ce-fa3b1492f823 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:55.864470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e9c7-3be4-4a88-b4ce-fa3b1492f823 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:55.864739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e9c7-3be4-4a88-b4ce-fa3b1492f823 None admin] PATH_INFO: `/v3/projects/8a87ea16ab0741c8b96b76059c8dd284` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:55.865893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34e9c7-3be4-4a88-b4ce-fa3b1492f823 None admin] RBAC: Authorizing `identity:get_project(project_id=8a87ea16ab0741c8b96b76059c8dd284)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:55.867308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34e9c7-3be4-4a88-b4ce-fa3b1492f823 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:55.869002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 589/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:55 2022] GET /identity/v3/projects/8a87ea16ab0741c8b96b76059c8dd284 => generated 361 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:55.877511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:55.976404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14b963-66b2-4067-a6f6-2044774075db None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:56.010742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65175a-aa40-402b-98da-2992010a7d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:56.011344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65175a-aa40-402b-98da-2992010a7d2f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:56.011623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65175a-aa40-402b-98da-2992010a7d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:56.011895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65175a-aa40-402b-98da-2992010a7d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:56.012609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65175a-aa40-402b-98da-2992010a7d2f None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=8a87ea16ab0741c8b96b76059c8dd284, user.id=9439fe2941104fccaad35d200fff8ea0) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:56.012969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65175a-aa40-402b-98da-2992010a7d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:56.014420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65175a-aa40-402b-98da-2992010a7d2f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:56.020504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 596/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:57:55 2022] GET /identity/v3/role_assignments?user.id=9439fe2941104fccaad35d200fff8ea0&scope.project.id=8a87ea16ab0741c8b96b76059c8dd284&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:57.910847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809b3d-b418-4a6d-b1ab-2b51bd2c427c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:57.911392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809b3d-b418-4a6d-b1ab-2b51bd2c427c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:57.911392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809b3d-b418-4a6d-b1ab-2b51bd2c427c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:57.912713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 590/1186] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:57:57.917917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815e96-77c0-4cfc-bd9d-59afe297f1a0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:57.918282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815e96-77c0-4cfc-bd9d-59afe297f1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:57.918478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815e96-77c0-4cfc-bd9d-59afe297f1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:57.936969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2e815e96-77c0-4cfc-bd9d-59afe297f1a0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:57:57.938181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2e815e96-77c0-4cfc-bd9d-59afe297f1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:57:57.996060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:58.005214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 597/1187] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:57:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:57:58.276567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc809b3d-b418-4a6d-b1ab-2b51bd2c427c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:58.384446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc809b3d-b418-4a6d-b1ab-2b51bd2c427c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:58.393459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:58.417679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:58.418299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:58.418581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:58.418856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:58.429343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:57:58.434437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:57:58.435589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:57:58.436450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 591/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:57:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:57:58.442113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e815e96-77c0-4cfc-bd9d-59afe297f1a0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:58.537918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e815e96-77c0-4cfc-bd9d-59afe297f1a0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:58.570806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:58.571405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:58.571689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:58.571967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:58.572635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:57:58.572941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:58.574334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:58.582736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 598/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:57:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:58.588421 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:58.683414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722559bc-b0fe-4518-86f6-d16fcb4cff27 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:58.715814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:58.716489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:58.723023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:58.723023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] PATH_INFO: `/v3/users/9439fe2941104fccaad35d200fff8ea0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:58.723023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] RBAC: Authorizing `identity:get_user(user_id=9439fe2941104fccaad35d200fff8ea0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:58.723023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:58.723023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 592/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:57:58 2022] GET /identity/v3/users/9439fe2941104fccaad35d200fff8ea0 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:58.726854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:58.782032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:58.825619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1046-70ae-4314-8a2d-0eb78771694a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:58.858375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b58c0-cdaf-401f-8782-1f3db7835060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:58.858995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b58c0-cdaf-401f-8782-1f3db7835060 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:58.859279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b58c0-cdaf-401f-8782-1f3db7835060 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:58.859553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b58c0-cdaf-401f-8782-1f3db7835060 None admin] PATH_INFO: `/v3/projects/8a87ea16ab0741c8b96b76059c8dd284` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:58.860660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b58c0-cdaf-401f-8782-1f3db7835060 None admin] RBAC: Authorizing `identity:get_project(project_id=8a87ea16ab0741c8b96b76059c8dd284)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:58.862121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b58c0-cdaf-401f-8782-1f3db7835060 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:58.864348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 599/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:57:58 2022] GET /identity/v3/projects/8a87ea16ab0741c8b96b76059c8dd284 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:57:58.870266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:57:58.905183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:57:58.966463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c18b02-cdba-48b9-8f39-714db187d931 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:57:58.999142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadc883-59a9-40c9-b79e-b852b9fa3ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:57:58.999839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc883-59a9-40c9-b79e-b852b9fa3ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:57:59.000118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc883-59a9-40c9-b79e-b852b9fa3ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:57:59.000389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc883-59a9-40c9-b79e-b852b9fa3ad2 None admin] PATH_INFO: `/v3/projects/8a87ea16ab0741c8b96b76059c8dd284/users/9439fe2941104fccaad35d200fff8ea0/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:57:59.002548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadc883-59a9-40c9-b79e-b852b9fa3ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a87ea16ab0741c8b96b76059c8dd284, user_id=9439fe2941104fccaad35d200fff8ea0, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:57:59.004636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadc883-59a9-40c9-b79e-b852b9fa3ad2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:57:59.093277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 593/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:57:58 2022] PUT /identity/v3/projects/8a87ea16ab0741c8b96b76059c8dd284/users/9439fe2941104fccaad35d200fff8ea0/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:58:00.960632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ac025-a023-4099-a878-e48b9596b7ef None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:00.961158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ac025-a023-4099-a878-e48b9596b7ef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:00.961252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ac025-a023-4099-a878-e48b9596b7ef None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:00.962548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 600/1193] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:58:00.967626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fdcf2-8cea-4a02-a542-f0f6f89f1c2e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:00.968009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fdcf2-8cea-4a02-a542-f0f6f89f1c2e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:00.968187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fdcf2-8cea-4a02-a542-f0f6f89f1c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:00.986496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-663fdcf2-8cea-4a02-a542-f0f6f89f1c2e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:58:00.987702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-663fdcf2-8cea-4a02-a542-f0f6f89f1c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:58:01.051469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 594/1194] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:58:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:58:01.319389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ac025-a023-4099-a878-e48b9596b7ef None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:58:01.417976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ac025-a023-4099-a878-e48b9596b7ef None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:58:01.440592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:58:01.452795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:58:01.458428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:01.458428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:01.458428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:01.464455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:58:01.469317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:58:01.470434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:58:01.471425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 601/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:58:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:58:01.477294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fdcf2-8cea-4a02-a542-f0f6f89f1c2e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:58:01.504244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:58:01.574601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fdcf2-8cea-4a02-a542-f0f6f89f1c2e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:58:01.606875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:58:01.607462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:01.607744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:01.608015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:01.608665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:58:01.608964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:58:01.610347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:58:01.618217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 595/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:58:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:58:01.623915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:58:01.719963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3468b9-63d4-4106-98ec-94d3bbb19ecd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:58:01.752079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:58:01.752686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:01.752962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:01.753236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] PATH_INFO: `/v3/users/9439fe2941104fccaad35d200fff8ea0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:01.754354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] RBAC: Authorizing `identity:get_user(user_id=9439fe2941104fccaad35d200fff8ea0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:58:01.756005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:58:01.757296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 602/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Sep 4 15:58:01 2022] GET /identity/v3/users/9439fe2941104fccaad35d200fff8ea0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:58:01.763214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:58:01.849421 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:58:01.860942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affabf0-814c-4c1d-b7cf-7f85c92aaab0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:58:01.893276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:58:01.893894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:01.894174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:01.895003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:01.959107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:58:01.960525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:58:02.042260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:58:02.053884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 596/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:58:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:58:02.059757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:58:02.111406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:58:02.156760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cf42d-a573-4cfd-b98b-b71e4805fc5e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:58:02.188409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b88157-52f4-47f8-8798-f1fc11679dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:58:02.189029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88157-52f4-47f8-8798-f1fc11679dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:02.189309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88157-52f4-47f8-8798-f1fc11679dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:02.189603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88157-52f4-47f8-8798-f1fc11679dc1 None admin] PATH_INFO: `/v3/projects/8a87ea16ab0741c8b96b76059c8dd284` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:02.190706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b88157-52f4-47f8-8798-f1fc11679dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=8a87ea16ab0741c8b96b76059c8dd284)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:58:02.192754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b88157-52f4-47f8-8798-f1fc11679dc1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:58:02.194384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 603/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Sep 4 15:58:02 2022] GET /identity/v3/projects/8a87ea16ab0741c8b96b76059c8dd284 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:58:02.200422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:58:02.296751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104d214-2df3-4c37-9a5e-d85e2d8c4ffe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:58:02.331187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ec124-094e-44e9-aea5-e37f4700b506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:58:02.331800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ec124-094e-44e9-aea5-e37f4700b506 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:58:02.332748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ec124-094e-44e9-aea5-e37f4700b506 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:58:02.333030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ec124-094e-44e9-aea5-e37f4700b506 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:58:02.333787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ec124-094e-44e9-aea5-e37f4700b506 None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=8a87ea16ab0741c8b96b76059c8dd284, user.id=9439fe2941104fccaad35d200fff8ea0) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:58:02.334551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ec124-094e-44e9-aea5-e37f4700b506 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:58:02.336118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ec124-094e-44e9-aea5-e37f4700b506 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:58:02.342939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 597/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:58:02 2022] GET /identity/v3/role_assignments?user.id=9439fe2941104fccaad35d200fff8ea0&scope.project.id=8a87ea16ab0741c8b96b76059c8dd284&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:58:12.184396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=40 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:58:12.340225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=74 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:13.882957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a2b1e-faf7-497f-9d5f-614db0469d30 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:13.883535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a2b1e-faf7-497f-9d5f-614db0469d30 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:13.883535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a2b1e-faf7-497f-9d5f-614db0469d30 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:13.884900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 604/1201] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:13.889859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7946b-5101-4353-ab33-467bb4bebea4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:13.890342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7946b-5101-4353-ab33-467bb4bebea4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:13.890460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7946b-5101-4353-ab33-467bb4bebea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:13.908493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c0a7946b-5101-4353-ab33-467bb4bebea4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:13.909790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c0a7946b-5101-4353-ab33-467bb4bebea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:13.977082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 598/1202] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:14.249414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a2b1e-faf7-497f-9d5f-614db0469d30 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:14.355613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a2b1e-faf7-497f-9d5f-614db0469d30 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:14.389366 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:14.390028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:14.390306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:14.390574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:14.402601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:14.404065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:14.419840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:14.424343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:14.425593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:14.425593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:14.425593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:14.425593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 605/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:14.427195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7946b-5101-4353-ab33-467bb4bebea4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:14.525026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7946b-5101-4353-ab33-467bb4bebea4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:14.552825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:14.558686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161c14e-27a5-496b-a6ac-6b6b878d482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:14.559555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161c14e-27a5-496b-a6ac-6b6b878d482a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:14.559809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161c14e-27a5-496b-a6ac-6b6b878d482a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:14.559922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161c14e-27a5-496b-a6ac-6b6b878d482a None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:14.560477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161c14e-27a5-496b-a6ac-6b6b878d482a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:14.560775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161c14e-27a5-496b-a6ac-6b6b878d482a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:14.562146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161c14e-27a5-496b-a6ac-6b6b878d482a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:14.573204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 599/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:14.579192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:14.633210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:14.680185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae94ca-1bee-40ff-ae94-24d6621023e8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:14.714592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947386a-03fb-400e-b2a6-159009bc8833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:14.715195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947386a-03fb-400e-b2a6-159009bc8833 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:14.715474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947386a-03fb-400e-b2a6-159009bc8833 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:14.715743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947386a-03fb-400e-b2a6-159009bc8833 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:14.716387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947386a-03fb-400e-b2a6-159009bc8833 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:14.717914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947386a-03fb-400e-b2a6-159009bc8833 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:14.810224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 606/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Sep 4 15:59:14 2022] POST /identity/v3/users => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:59:16.770808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7248-a836-4528-b775-c2d5570dfeba None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:16.771308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7248-a836-4528-b775-c2d5570dfeba None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:16.771308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7248-a836-4528-b775-c2d5570dfeba None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:16.772613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 600/1206] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:16.777803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a5a34-3f07-4d55-ae31-b4e65b016745 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:16.778223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a5a34-3f07-4d55-ae31-b4e65b016745 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:16.778352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a5a34-3f07-4d55-ae31-b4e65b016745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:16.796256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a10a5a34-3f07-4d55-ae31-b4e65b016745 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:16.797435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a10a5a34-3f07-4d55-ae31-b4e65b016745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:16.862358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 607/1207] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:17.137412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c7248-a836-4528-b775-c2d5570dfeba None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:17.239597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c7248-a836-4528-b775-c2d5570dfeba None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:17.272968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:17.273608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:17.273901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:17.274175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:17.284018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:59:17.289523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:17.290530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:17.291452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 601/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:59:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:17.296631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a5a34-3f07-4d55-ae31-b4e65b016745 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:17.336004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:17.395265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a5a34-3f07-4d55-ae31-b4e65b016745 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:17.428997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:17.429627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:17.429910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:17.430185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:17.430844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:17.431144 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:17.432518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:17.440053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 608/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:59:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:17.445794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:17.495644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:17.544324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d387d-82e7-44d0-b64e-1f10c5e874e9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:17.576044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:17.576660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:17.576937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:17.577204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:17.588332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:17.589877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:17.605870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:17.607610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:17.608559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:17.608559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:17.608559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:17.608559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 602/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:17.613055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:17.711084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:17.713932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71a24-6945-4b67-a5fe-99e9a913421e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:17.755107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:17.755729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:17.756012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:17.756290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:17.756937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:17.757241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:17.758642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:17.770957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 609/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:17.776758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:17.874530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20a38e-b9f0-40f3-aff1-f230ac547a4d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:17.876005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:17.909003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:17.909687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:17.909993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:17.910256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:17.917214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:17.918686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:59:17.930350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:17.931440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:17.932424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 603/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:17.937451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:18.038695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18a08e-301f-4215-8228-9fcf2120271a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:18.071924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:18.072519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:18.072795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:18.073096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:18.073845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:18.074151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:18.078813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:18.093250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 610/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Sep 4 15:59:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:18.099463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:18.199821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d42d3-d50c-415b-88d0-d4dfe497565b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:18.232170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:18.232769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:18.233048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:18.233319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:18.263058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:18.299203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:18.301157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:18.397316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 604/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:59:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 15:59:18.403283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:18.422113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:18.502783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023a5d-5ee1-4a2d-b6e8-2f80f31d0fb9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:18.536050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:18.536670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:18.536947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:18.537217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:18.546974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:18.548415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:18.563678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:18.564898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:18.566108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:18.566108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:18.566108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:18.566108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 611/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:18.570636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:18.630086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:18.669815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8c6fb-2e57-4fbb-a5ff-28a4bbf2be12 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:18.703433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:18.704035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:18.704316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:18.704589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:18.705227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:18.705529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:18.707377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:18.719101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 605/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:18.725438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:18.793798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:18.823645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1138-344e-487d-992d-0a86fe686e70 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:18.856945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:18.857586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:18.857880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:18.858154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:18.865699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:18.867134 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:59:18.879064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:18.880220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:18.881297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 612/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:18.886085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:18.989386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b184e-1414-4c4f-b25b-7ec606e280c5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:19.012182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:19.023741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02929c8-44b1-461e-81e2-7d0c0c13a91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:19.024339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02929c8-44b1-461e-81e2-7d0c0c13a91c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:19.024615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02929c8-44b1-461e-81e2-7d0c0c13a91c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:19.024886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02929c8-44b1-461e-81e2-7d0c0c13a91c None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:19.025589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02929c8-44b1-461e-81e2-7d0c0c13a91c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:19.025894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02929c8-44b1-461e-81e2-7d0c0c13a91c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:19.027301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02929c8-44b1-461e-81e2-7d0c0c13a91c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:19.038712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 606/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:19.044798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:19.142771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa403051-0c56-46f3-bfaf-3c954d99ddf8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:19.172190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:19.178995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7910f14-5b2a-40ac-b035-3df715f44863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:19.179592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7910f14-5b2a-40ac-b035-3df715f44863 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:19.179873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7910f14-5b2a-40ac-b035-3df715f44863 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:19.180147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7910f14-5b2a-40ac-b035-3df715f44863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:19.180865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7910f14-5b2a-40ac-b035-3df715f44863 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=07e04ce012c849baa403dda571f7454a) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:19.181173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7910f14-5b2a-40ac-b035-3df715f44863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:19.182745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7910f14-5b2a-40ac-b035-3df715f44863 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:19.188459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 613/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:59:19 2022] GET /identity/v3/role_assignments?user.id=07e04ce012c849baa403dda571f7454a&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:21.116889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191b863-2ee3-4106-ad91-b5510da2a4b2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:21.117452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191b863-2ee3-4106-ad91-b5510da2a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:21.117452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191b863-2ee3-4106-ad91-b5510da2a4b2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:21.118834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 607/1220] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:21.124023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315991-f9f9-46d5-811e-122cc6cecf1c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:21.127538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315991-f9f9-46d5-811e-122cc6cecf1c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:21.127538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315991-f9f9-46d5-811e-122cc6cecf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:21.143006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-96315991-f9f9-46d5-811e-122cc6cecf1c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:21.144755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-96315991-f9f9-46d5-811e-122cc6cecf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:21.212234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 614/1221] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:21.488803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191b863-2ee3-4106-ad91-b5510da2a4b2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:21.589933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191b863-2ee3-4106-ad91-b5510da2a4b2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:21.622774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:21.623392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:21.623671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:21.623944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:21.634928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:59:21.639764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:21.640960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:21.642055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 608/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:59:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:21.651338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96315991-f9f9-46d5-811e-122cc6cecf1c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:21.747725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96315991-f9f9-46d5-811e-122cc6cecf1c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:21.771074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:21.781481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:21.784282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:21.784282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:21.784282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:21.784282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:21.784282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:21.785193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:21.792829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 615/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:59:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:21.798777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:21.842265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:21.896365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bda22-56c0-4fab-812d-e3c366a28f1a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:21.928870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:21.929488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:21.929798 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:21.930098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:21.939771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:21.941965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:21.956071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:21.957191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:21.958337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:21.958337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:21.958337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:21.958337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 609/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:21.963246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:22.063925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4eb86-9ff3-4c40-ad90-5c8013e4fb84 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:22.099814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:22.100453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:22.100736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:22.101012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:22.101686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:22.101993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:22.103393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:22.117458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 616/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:22.123635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:22.219569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:22.226835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdc9ed-094e-4448-913a-f54bac20ed94 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:22.260908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:22.261525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:22.261836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:22.262111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:22.268852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:22.270752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:59:22.282128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:22.283344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:22.284900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 610/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:22.289737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:22.344091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:22.398106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c72144-1ae6-4e40-b328-c669d35782a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:22.433438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:22.434107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:22.434390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:22.434665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:22.435391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:22.435729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:22.437162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:22.454623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 617/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Sep 4 15:59:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:22.461356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:22.562108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cbbd4-432f-4391-b89a-a187368dc00a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:22.595728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:22.596284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:22.596563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:22.596832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:22.608210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:22.609658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:22.624600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:22.625842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:22.627209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:22.627209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:22.627209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:22.627209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 611/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:22.631977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:22.732136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03943f9-e269-4446-98dc-cefa82ace6d6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:22.734452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:22.765504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:22.766137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:22.766428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:22.766702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:22.767341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:22.767650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:22.769728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:22.783957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 618/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:22.790361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:22.800215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:22.889340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8215-6bf6-4684-ae6f-e596cf1ac79f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:22.923258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:22.923945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:22.924270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:22.924544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:22.931453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:22.932910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:59:22.944201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:22.951578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:22.952755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 612/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:22.953376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:23.050594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648309ee-e28e-40c4-b8ae-40bb4994d2e4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:23.087406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92f29e-6f86-4a01-b730-ec22a400a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:23.088019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92f29e-6f86-4a01-b730-ec22a400a14e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:23.088298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92f29e-6f86-4a01-b730-ec22a400a14e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:23.088568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92f29e-6f86-4a01-b730-ec22a400a14e None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:23.089257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92f29e-6f86-4a01-b730-ec22a400a14e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:23.089556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92f29e-6f86-4a01-b730-ec22a400a14e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:23.091012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92f29e-6f86-4a01-b730-ec22a400a14e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:23.103770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 619/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:23.109988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:23.175942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:23.209462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2309cb-09dc-4aa3-97f8-62ba497ef4fd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:23.242595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9a9d-3e9e-4e74-a618-61eea4bb66f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:23.243242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9a9d-3e9e-4e74-a618-61eea4bb66f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:23.243520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9a9d-3e9e-4e74-a618-61eea4bb66f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:23.243790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9a9d-3e9e-4e74-a618-61eea4bb66f0 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/07e04ce012c849baa403dda571f7454a/roles/fa94d66c831d468f83e224236c343293` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:23.258683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb9a9d-3e9e-4e74-a618-61eea4bb66f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=07e04ce012c849baa403dda571f7454a, role_id=fa94d66c831d468f83e224236c343293)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:23.260147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb9a9d-3e9e-4e74-a618-61eea4bb66f0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:23.319999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 613/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:59:23 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/07e04ce012c849baa403dda571f7454a/roles/fa94d66c831d468f83e224236c343293 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:59:25.191409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab4df-4253-4400-834a-c6ad02e9f3c8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:25.192032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab4df-4253-4400-834a-c6ad02e9f3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:25.192032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab4df-4253-4400-834a-c6ad02e9f3c8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:25.193340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 620/1233] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:25.198529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e0357-b5cf-4791-a4e0-7b658bb81df4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:25.198942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e0357-b5cf-4791-a4e0-7b658bb81df4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:25.199086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e0357-b5cf-4791-a4e0-7b658bb81df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:25.217957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-578e0357-b5cf-4791-a4e0-7b658bb81df4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:25.219137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-578e0357-b5cf-4791-a4e0-7b658bb81df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:25.285197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 614/1234] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:25.553345 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ab4df-4253-4400-834a-c6ad02e9f3c8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:25.577948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:25.653092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ab4df-4253-4400-834a-c6ad02e9f3c8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:25.687800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:25.688425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:25.688708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:25.688980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:25.698768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:59:25.703572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:25.704712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:25.705641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 621/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Sep 4 15:59:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:25.713391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e0357-b5cf-4791-a4e0-7b658bb81df4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:25.760062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:25.811475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e0357-b5cf-4791-a4e0-7b658bb81df4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:25.845205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:25.845828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:25.846110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:25.846383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:25.847681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:25.848072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:25.849467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:25.857447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 615/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Sep 4 15:59:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:59:25.863282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:25.946664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:25.962815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83002cc-b04e-408f-8f4b-4bfc29c3db91 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:25.996817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:25.997449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:25.997763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:25.998042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:26.008871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:26.010347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:26.025301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:26.026495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:26.027846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:26.027846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:26.027846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:26.027846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 622/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:26.032369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:26.131987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757cf59-33d2-4866-b063-4afb5f45cb89 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:26.140604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:26.164874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:26.165472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:26.165779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:26.166059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:26.166701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:26.167006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:26.168364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:26.179793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 616/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:26.185750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:26.285918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e023-91f3-4fdd-b45d-e32aa02862cc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:26.318250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:26.318872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:26.319147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:26.319417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:26.322162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:26.327088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:26.328507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:59:26.340187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:26.341259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:26.342382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 623/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:26.347117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:26.450603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95ea64-d14e-4154-ad9b-9955a92a2487 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:26.483720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:26.484364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:26.484656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:26.484926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:26.485677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:26.485981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:26.487399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:26.506371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 617/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Sep 4 15:59:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:26.512288 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:26.611789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55355b97-f176-4fad-bd7a-1daf20597e1e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:26.645540 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:26.646167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:26.646447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:26.646718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:26.716175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:26.723430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:26.724909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:26.827598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 624/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:59:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:59:26.834145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:26.860612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:26.930862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ebe45-e83d-4a0c-b27e-144f6faa3e28 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:26.967565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:26.968127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:26.968408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:26.968681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:26.979271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:26.982675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:26.995273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:26.996282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:26.997260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:26.997260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:26.997260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:26.997260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 618/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:27.002360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:27.098460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:27.104111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6ba93-86a8-489f-86a0-f80e12d29876 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:27.138453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:27.139056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:27.139333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:27.139603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:27.140236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:27.140537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:27.141927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:27.152742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 625/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:27.158624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:27.239811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:27.260986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096fdc6-6088-4e45-9708-d08cef47632b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:27.292600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:27.293223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:27.293501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:27.293801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:27.300746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:27.302211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:59:27.313141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:27.315604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:27.317243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 619/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:59:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:27.320951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:27.420845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399cb9f-1f7c-4cb5-9f85-664c550f0143 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:27.454688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9840-160e-498e-86cb-3d2f9a4a751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:27.455293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9840-160e-498e-86cb-3d2f9a4a751f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:27.455578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9840-160e-498e-86cb-3d2f9a4a751f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:27.455855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9840-160e-498e-86cb-3d2f9a4a751f None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:27.456774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9840-160e-498e-86cb-3d2f9a4a751f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:27.457080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9840-160e-498e-86cb-3d2f9a4a751f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:27.458553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9840-160e-498e-86cb-3d2f9a4a751f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:27.470323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 626/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:59:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:27.477209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:27.576789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8bafd-7f25-4cc2-808b-a8803c1326a5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:27.609492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13461f5f-d1e2-447d-81a7-e40c9267ca53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:27.610126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13461f5f-d1e2-447d-81a7-e40c9267ca53 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:27.610409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13461f5f-d1e2-447d-81a7-e40c9267ca53 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:27.610684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13461f5f-d1e2-447d-81a7-e40c9267ca53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:27.611395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13461f5f-d1e2-447d-81a7-e40c9267ca53 None admin] RBAC: Adding query filter params (role.id=fa94d66c831d468f83e224236c343293, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=07e04ce012c849baa403dda571f7454a) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:27.611703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13461f5f-d1e2-447d-81a7-e40c9267ca53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:27.613810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13461f5f-d1e2-447d-81a7-e40c9267ca53 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:27.619434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 620/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:59:27 2022] GET /identity/v3/role_assignments?user.id=07e04ce012c849baa403dda571f7454a&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=fa94d66c831d468f83e224236c343293 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:29.598611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76c7d-d011-46e9-b7a0-23e2aa2acdaa None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:29.599122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76c7d-d011-46e9-b7a0-23e2aa2acdaa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:29.599122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76c7d-d011-46e9-b7a0-23e2aa2acdaa None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:29.600491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 627/1247] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:29.605877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456daf0-66b1-41c3-a91e-26d3ee3582d2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:29.606279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456daf0-66b1-41c3-a91e-26d3ee3582d2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:29.606444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456daf0-66b1-41c3-a91e-26d3ee3582d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:29.612671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:29.624950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6456daf0-66b1-41c3-a91e-26d3ee3582d2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:29.626158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6456daf0-66b1-41c3-a91e-26d3ee3582d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:29.694848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 621/1248] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:29.971644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e76c7d-d011-46e9-b7a0-23e2aa2acdaa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:29.991191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:30.073680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e76c7d-d011-46e9-b7a0-23e2aa2acdaa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:30.107806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:30.108429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:30.108704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:30.108973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:30.119263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:59:30.124050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:30.125155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:30.126347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 628/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:59:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:30.131921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456daf0-66b1-41c3-a91e-26d3ee3582d2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:30.229629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456daf0-66b1-41c3-a91e-26d3ee3582d2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:30.257483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:30.262106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:30.262721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:30.263086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:30.263363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:30.264014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:30.264312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:30.265687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:30.274205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 622/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:30.279995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:30.358461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:30.380578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce23c7f-1f10-4380-94f5-65f7811db469 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:30.414196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:30.414819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:30.415097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:30.415366 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:30.424913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:30.426563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:30.440791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:30.442065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:30.443282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:30.443282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:30.443282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:30.443282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 629/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:30.447870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:30.546660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075bc1a-8ad6-4e9c-8d83-610ac2505cf3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:30.578968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:30.579565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:30.579850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:30.580113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:30.580746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:30.581051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:30.583425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:30.595156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 623/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:30.602052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:30.702384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574b279-7e1b-402f-a01e-ac6f854f06e2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:30.728355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:30.736149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:30.736781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:30.737063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:30.737379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:30.745344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:30.746858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:59:30.758705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:30.768032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:30.769886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 630/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:59:30.770662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:30.821092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:30.865211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bee6-ffe0-48c9-bf90-1ce87246316b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:30.899422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:30.900026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:30.900307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:30.900580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:30.901312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:30.901640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:30.903050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:30.920420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 624/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Sep 4 15:59:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:30.927111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:31.026051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9a0e-adef-4f06-9948-68e94eb76261 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:31.059064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:31.059781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:31.060058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:31.060329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:31.114171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:31.125983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:31.127688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:31.224458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 631/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:59:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:59:31.230390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:31.330692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af3a88-a2be-4df6-bb9c-83fde18377c9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:31.344798 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:31.366547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:31.367103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:31.367378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:31.367642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:31.377774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:31.379236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:31.393344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:31.394543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:31.395461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:31.395461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:31.395461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:31.395461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 625/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:31.400355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:31.484829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:31.496984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588fe63-71a8-462b-9be9-536234e0210a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:31.530254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:31.530851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:31.531129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:31.531398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:31.532029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:31.532325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:31.533699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:31.545122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 632/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:31.551656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:31.652533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b51c0-614f-4ec8-88ee-264429785659 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:31.685322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:31.686034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:31.686317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:31.686592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:31.693908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:31.695080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:59:31.707583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:31.708668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:31.709942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 626/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:59:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:59:31.714618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:31.814497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c457-c401-4492-9982-0be116f9ff72 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:31.848400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2c9fa-d297-4c7c-8c74-ca1275a98a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:31.848999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c9fa-d297-4c7c-8c74-ca1275a98a6a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:31.849279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c9fa-d297-4c7c-8c74-ca1275a98a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:31.849552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c9fa-d297-4c7c-8c74-ca1275a98a6a None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:31.850259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2c9fa-d297-4c7c-8c74-ca1275a98a6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:31.850565 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2c9fa-d297-4c7c-8c74-ca1275a98a6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:31.851989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2c9fa-d297-4c7c-8c74-ca1275a98a6a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:31.863783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 633/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:59:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:31.869950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:31.896430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:31.974975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24051261-2792-4638-b3bd-8971a5fa5ce1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:32.008495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05757a1-119f-49a1-ad60-b0ae5868df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:32.009093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05757a1-119f-49a1-ad60-b0ae5868df0d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:32.009370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05757a1-119f-49a1-ad60-b0ae5868df0d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:32.009670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05757a1-119f-49a1-ad60-b0ae5868df0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:32.010474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05757a1-119f-49a1-ad60-b0ae5868df0d None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=07e04ce012c849baa403dda571f7454a) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:32.010779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05757a1-119f-49a1-ad60-b0ae5868df0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:32.012187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05757a1-119f-49a1-ad60-b0ae5868df0d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:32.017501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 627/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:59:31 2022] GET /identity/v3/role_assignments?user.id=07e04ce012c849baa403dda571f7454a&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:33.908268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951af8-67c6-4ac1-a372-a6647c2353ca None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:33.908857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951af8-67c6-4ac1-a372-a6647c2353ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:33.908857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951af8-67c6-4ac1-a372-a6647c2353ca None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:33.910250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 634/1261] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:33.915477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006596f7-cfa9-4a6b-a8c9-6ae60d62b298 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:33.915929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006596f7-cfa9-4a6b-a8c9-6ae60d62b298 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:33.916022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006596f7-cfa9-4a6b-a8c9-6ae60d62b298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:33.934762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-006596f7-cfa9-4a6b-a8c9-6ae60d62b298 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:33.936012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-006596f7-cfa9-4a6b-a8c9-6ae60d62b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:34.001109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 628/1262] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:34.277906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2951af8-67c6-4ac1-a372-a6647c2353ca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:34.289069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:34.378033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2951af8-67c6-4ac1-a372-a6647c2353ca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:34.412210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:34.412851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:34.413130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:34.413401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:34.423737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:59:34.429476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:34.433066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:34.433957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 635/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:34.437318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006596f7-cfa9-4a6b-a8c9-6ae60d62b298 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:34.460521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:34.539025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006596f7-cfa9-4a6b-a8c9-6ae60d62b298 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:34.572184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:34.572776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:34.573050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:34.573318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:34.573996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:34.574295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:34.575749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:34.585182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 629/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:34.591047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:34.658593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:34.691432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bc1c-6efc-46e9-8034-f2ccad1b319d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:34.723110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:34.723726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:34.724006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:34.724278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:34.734571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:34.736037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:34.751258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:34.752769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:34.754095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:34.754095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:34.754095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:34.754095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 636/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:34.759032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:34.839849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:34.857708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9b83f-01b8-4da3-b750-49897b42acc8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:34.890836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:34.891428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:34.891704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:34.891972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:34.892603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:34.892900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:34.894316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:34.904890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 630/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:34.910774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:35.011473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1d054-1536-4311-a3f9-bd7c19cb946b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:35.033077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:35.054456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:35.055093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:35.055377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:35.055654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:35.063267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:35.064713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:59:35.075816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:35.078215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:35.079438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 637/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:35.085177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:35.188822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99f42-6837-4197-8d96-4f4990ec82fb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:35.222083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:35.222796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:35.223075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:35.223350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:35.224078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:35.224383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:35.225840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:35.234606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:35.243062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 631/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Sep 4 15:59:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:35.248808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:35.350762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce9b87-627a-4438-9360-df6b63537e66 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:35.385895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:35.386523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:35.386813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:35.387084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:35.397894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:35.399339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:35.413788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:35.415887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:35.417018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:35.417018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:35.417018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:35.417018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 638/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:35.420736 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:35.520228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6d133-8dda-46d4-81cb-bc1d752e1b59 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:35.555156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:35.555742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:35.556025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:35.556301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:35.556939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:35.557243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:35.558647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:35.569761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 632/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:35.575438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:35.607346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:35.674236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0614e9-d264-46d7-b77d-ecfd70d7a852 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:35.711546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:35.712181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:35.712546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:35.712787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:35.719348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:35.720794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:59:35.733217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:35.734396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:35.735422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 639/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:35.740270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:35.798445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:35.838662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eff1b-f02e-4274-98e4-22f4f6f58830 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:35.871434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c655f77-b800-4327-b1b1-5da76f23a721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:35.872040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c655f77-b800-4327-b1b1-5da76f23a721 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:35.872369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c655f77-b800-4327-b1b1-5da76f23a721 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:35.872648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c655f77-b800-4327-b1b1-5da76f23a721 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:35.873336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c655f77-b800-4327-b1b1-5da76f23a721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:35.873667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c655f77-b800-4327-b1b1-5da76f23a721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:35.875081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c655f77-b800-4327-b1b1-5da76f23a721 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:35.887818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 633/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:35.894703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:35.983346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:35.991975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070341-cbb1-4db6-83d1-4aee99351d9a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:36.025676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee09964-3261-4ea6-a2e8-3d693fd957c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:36.026321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee09964-3261-4ea6-a2e8-3d693fd957c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:36.026599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee09964-3261-4ea6-a2e8-3d693fd957c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:36.026870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee09964-3261-4ea6-a2e8-3d693fd957c9 None admin] PATH_INFO: `/v3/projects/d9b51abd49f54c64af586d43190b826e/users/07e04ce012c849baa403dda571f7454a/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:36.028734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee09964-3261-4ea6-a2e8-3d693fd957c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b51abd49f54c64af586d43190b826e, user_id=07e04ce012c849baa403dda571f7454a, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:36.030195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee09964-3261-4ea6-a2e8-3d693fd957c9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:36.125516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 640/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Sep 4 15:59:35 2022] PUT /identity/v3/projects/d9b51abd49f54c64af586d43190b826e/users/07e04ce012c849baa403dda571f7454a/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 15:59:38.055102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5088d4-5b09-4453-832c-bdde84c124a9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:38.055633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5088d4-5b09-4453-832c-bdde84c124a9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:38.055726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5088d4-5b09-4453-832c-bdde84c124a9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:38.057030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 634/1274] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:38.062231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfd478-ed20-448c-b2ab-b57eda685055 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:38.062632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfd478-ed20-448c-b2ab-b57eda685055 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:38.062792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfd478-ed20-448c-b2ab-b57eda685055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:38.081838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-70bfd478-ed20-448c-b2ab-b57eda685055 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:38.083056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70bfd478-ed20-448c-b2ab-b57eda685055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:38.150815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 641/1275] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:38.429904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5088d4-5b09-4453-832c-bdde84c124a9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:38.533356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5088d4-5b09-4453-832c-bdde84c124a9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:38.551229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:38.567635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:38.568248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:38.568524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:38.568794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:38.578607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 15:59:38.583910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:38.585146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:38.593246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 635/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Sep 4 15:59:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:38.594199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfd478-ed20-448c-b2ab-b57eda685055 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:38.672548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:38.689574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfd478-ed20-448c-b2ab-b57eda685055 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:38.722891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:38.723497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:38.723780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:38.724057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:38.724719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:38.725023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:38.726439 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:38.734271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 642/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Sep 4 15:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:38.740254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:38.843125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd721c1-9603-418b-ad16-a7036cf3175c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:38.876170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:38.876792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:38.877073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:38.877665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:38.888513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:38.894020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:38.905309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:38.906454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:38.907400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:38.907400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:38.907400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:38.907400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 636/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:59:38.912526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:39.014239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799960-0ada-46a4-ad8b-caea96e1027c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:39.047557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:39.048158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:39.048434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:39.048704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:39.049336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:39.049716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:39.051230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:39.063153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 643/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:39.071673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:39.102958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:39.173866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629183e-8c66-467b-a06f-32f12fe15c94 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:39.208531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:39.209177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:39.209535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:39.209836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:39.216931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:39.218386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 15:59:39.230237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:39.231386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:39.232442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 637/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:39.238111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:39.242273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:39.342550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd912d-35b2-4a98-ac3f-40ba74071fb4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:39.380009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:39.382253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:39.382537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:39.382815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:39.383604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:39.383919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:39.387960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:39.404905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 644/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Sep 4 15:59:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:39.410896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:39.504578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:39.513734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b1db2-29b8-4e1e-bbd8-efbc2e4ab4bb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:39.546465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:39.547056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:39.547333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:39.547604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:39.614696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:39.616175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:39.710671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:39.720369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 638/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Sep 4 15:59:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 15:59:39.728106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:39.792157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:39.829655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc69d2-64d1-48c9-b903-6d6d7048c8c2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:39.864398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:39.865016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:39.865293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:39.865577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:39.876889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:39.879422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 15:59:39.894522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:39.895612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 15:59:39.896743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 15:59:39.896743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 15:59:39.896743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:39.896743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 645/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 15:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:39.901506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:40.000135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14994b-816a-48f7-9447-8f9a2c70c577 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:40.038380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:40.039006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:40.039283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:40.039552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:40.040212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:40.040514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:40.041929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:40.054643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 639/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Sep 4 15:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:40.060408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:40.170104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940108-9fe7-45b9-abee-adf315f2c109 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:40.185907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:40.208611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:40.209267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:40.209545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:40.209845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:40.217179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:40.218668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 15:59:40.233369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:40.234601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Sep 04 15:59:40.235873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 646/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Sep 4 15:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 15:59:40.240880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:40.297270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:40.352765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb02a-c385-4246-8aed-d9bcf09949da None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:40.388252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b5ac2-4d2c-4c8f-8046-6a04a5de0dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:40.388859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b5ac2-4d2c-4c8f-8046-6a04a5de0dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:40.389140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b5ac2-4d2c-4c8f-8046-6a04a5de0dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:40.389410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b5ac2-4d2c-4c8f-8046-6a04a5de0dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:40.390187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b5ac2-4d2c-4c8f-8046-6a04a5de0dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:40.390497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b5ac2-4d2c-4c8f-8046-6a04a5de0dc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:40.391923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b5ac2-4d2c-4c8f-8046-6a04a5de0dc4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:40.404583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 640/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Sep 4 15:59:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 15:59:40.411594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:40.521724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c920e-5ae6-4d3f-806c-0c3f1d1c20aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:40.554882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553a7d-9571-4b4a-a039-2b85c468ef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:40.555427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553a7d-9571-4b4a-a039-2b85c468ef94 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:40.555707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553a7d-9571-4b4a-a039-2b85c468ef94 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:40.555980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553a7d-9571-4b4a-a039-2b85c468ef94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:40.556700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553a7d-9571-4b4a-a039-2b85c468ef94 None admin] RBAC: Adding query filter params (role.id=09a611f233ce4fff97af2ed34e916c22, scope.project.id=d9b51abd49f54c64af586d43190b826e, user.id=07e04ce012c849baa403dda571f7454a) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:40.557008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553a7d-9571-4b4a-a039-2b85c468ef94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:40.558464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553a7d-9571-4b4a-a039-2b85c468ef94 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:40.563987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 647/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Sep 4 15:59:40 2022] GET /identity/v3/role_assignments?user.id=07e04ce012c849baa403dda571f7454a&scope.project.id=d9b51abd49f54c64af586d43190b826e&role.id=09a611f233ce4fff97af2ed34e916c22 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:42.496840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ca69e-ae73-4aa8-893e-b61b77cccb1c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:42.497360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ca69e-ae73-4aa8-893e-b61b77cccb1c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:42.497360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ca69e-ae73-4aa8-893e-b61b77cccb1c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:42.498729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 641/1288] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:42.504626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987ed2-0ff1-49cd-b06f-68a58b5d43d0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:42.505004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987ed2-0ff1-49cd-b06f-68a58b5d43d0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:42.505190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987ed2-0ff1-49cd-b06f-68a58b5d43d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:42.523863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f6987ed2-0ff1-49cd-b06f-68a58b5d43d0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:42.525070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f6987ed2-0ff1-49cd-b06f-68a58b5d43d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:42.554768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:42.592219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 648/1289] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:42.859387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ca69e-ae73-4aa8-893e-b61b77cccb1c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:42.962040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ca69e-ae73-4aa8-893e-b61b77cccb1c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:42.973493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:42.995855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:42.996473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:42.996748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:42.997014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:42.997608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:43.002508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:43.007709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:43.008886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:43.009964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 642/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:59:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:43.014745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6987ed2-0ff1-49cd-b06f-68a58b5d43d0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:43.111777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6987ed2-0ff1-49cd-b06f-68a58b5d43d0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:43.144959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b7916-5bb0-49bc-aef0-688d27ec14c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:43.145599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b7916-5bb0-49bc-aef0-688d27ec14c4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:43.145898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b7916-5bb0-49bc-aef0-688d27ec14c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:43.146171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b7916-5bb0-49bc-aef0-688d27ec14c4 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:43.146800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b7916-5bb0-49bc-aef0-688d27ec14c4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:43.147105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b7916-5bb0-49bc-aef0-688d27ec14c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:43.148481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b7916-5bb0-49bc-aef0-688d27ec14c4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:43.153195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 649/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Sep 4 15:59:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:43.159023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:43.258518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfa25c-ac52-4d61-8925-e2f99c187c32 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:43.291086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a1c70-17ed-4126-9e48-d24fc5154ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:43.291679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a1c70-17ed-4126-9e48-d24fc5154ede None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:43.291958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a1c70-17ed-4126-9e48-d24fc5154ede None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:43.292227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a1c70-17ed-4126-9e48-d24fc5154ede None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:43.292848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a1c70-17ed-4126-9e48-d24fc5154ede None admin] RBAC: Adding query filter params (type=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:43.293148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a1c70-17ed-4126-9e48-d24fc5154ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:43.294518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a1c70-17ed-4126-9e48-d24fc5154ede None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:43.298844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 643/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Sep 4 15:59:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:45.233722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8a60a-7436-4514-a471-124606c651e8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:45.234250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8a60a-7436-4514-a471-124606c651e8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:45.234250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8a60a-7436-4514-a471-124606c651e8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:45.235622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 650/1293] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:45.241314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fcd64-6214-42bb-8bb9-a94d6db304e6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:45.246720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fcd64-6214-42bb-8bb9-a94d6db304e6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:45.246720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fcd64-6214-42bb-8bb9-a94d6db304e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:45.260664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-701fcd64-6214-42bb-8bb9-a94d6db304e6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:45.261900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-701fcd64-6214-42bb-8bb9-a94d6db304e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:45.290827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:45.329782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 644/1294] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:45.601442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8a60a-7436-4514-a471-124606c651e8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:45.632867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:45.701739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8a60a-7436-4514-a471-124606c651e8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:45.735521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d30f-5d09-4242-927d-9b77ae25ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:45.736118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d30f-5d09-4242-927d-9b77ae25ce4c None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:45.736395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d30f-5d09-4242-927d-9b77ae25ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:45.736662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d30f-5d09-4242-927d-9b77ae25ce4c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:45.737324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270d30f-5d09-4242-927d-9b77ae25ce4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:45.738710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270d30f-5d09-4242-927d-9b77ae25ce4c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:45.810740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 651/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Sep 4 15:59:45 2022] POST /identity/v3/services => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 15:59:47.705373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057005b-4c96-45fb-9205-2404bf703192 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:47.706486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057005b-4c96-45fb-9205-2404bf703192 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:47.706486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057005b-4c96-45fb-9205-2404bf703192 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:47.707868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 645/1296] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:47.712928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd95d9-062e-4c86-a22d-95951ddbd0ec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:47.713368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd95d9-062e-4c86-a22d-95951ddbd0ec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:47.713476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd95d9-062e-4c86-a22d-95951ddbd0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:47.732557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f4dd95d9-062e-4c86-a22d-95951ddbd0ec None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:47.733781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f4dd95d9-062e-4c86-a22d-95951ddbd0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:47.807140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 652/1297] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:47 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 15:59:48.078221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057005b-4c96-45fb-9205-2404bf703192 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:48.178988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057005b-4c96-45fb-9205-2404bf703192 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:48.211495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:48.212112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:48.212388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:48.212661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:48.213232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:48.214642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:48.223883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:48.225243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:48.235242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 646/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:59:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:48.236269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd95d9-062e-4c86-a22d-95951ddbd0ec None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:48.249269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:48.328605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd95d9-062e-4c86-a22d-95951ddbd0ec None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:48.362437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:48.363044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:48.363322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:48.363591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:48.364215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:48.364513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:48.365910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:48.370869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 653/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Sep 4 15:59:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:48.376838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:48.410921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:48.478606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2eec3-0795-4d34-804d-5cf606cb6669 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:48.512721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3849dfc-6115-4702-bf3d-7d0343786708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:48.513321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849dfc-6115-4702-bf3d-7d0343786708 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:48.513624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849dfc-6115-4702-bf3d-7d0343786708 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:48.513899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849dfc-6115-4702-bf3d-7d0343786708 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:48.514596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3849dfc-6115-4702-bf3d-7d0343786708 None admin] RBAC: Adding query filter params (interface=public, service_id=267bf3f6a4ff472da4cbd4074c18ce9d, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:48.514903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3849dfc-6115-4702-bf3d-7d0343786708 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:48.516261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3849dfc-6115-4702-bf3d-7d0343786708 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:48.521841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 647/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Sep 4 15:59:48 2022] GET /identity/v3/endpoints?service_id=267bf3f6a4ff472da4cbd4074c18ce9d&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:48.527650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:48.607825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:48.624954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e3651-b57e-4fa2-87d4-b44ef1a27f5a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:48.665121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347cac9-d519-4960-ada8-aa6f4b6d80b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:48.665754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347cac9-d519-4960-ada8-aa6f4b6d80b2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:48.666037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347cac9-d519-4960-ada8-aa6f4b6d80b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:48.666347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347cac9-d519-4960-ada8-aa6f4b6d80b2 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:48.666946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347cac9-d519-4960-ada8-aa6f4b6d80b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:48.668300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347cac9-d519-4960-ada8-aa6f4b6d80b2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:48.675044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 654/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Sep 4 15:59:48 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 15:59:50.571008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852acf7a-ca8a-4b1b-a40b-ac514f4875f2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:50.571556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852acf7a-ca8a-4b1b-a40b-ac514f4875f2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:50.571556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852acf7a-ca8a-4b1b-a40b-ac514f4875f2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:50.573035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 648/1302] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 15:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 15:59:50.578492 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48719c-3691-4557-aab0-f44690c6ae2c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:50.578941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48719c-3691-4557-aab0-f44690c6ae2c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:50.579037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48719c-3691-4557-aab0-f44690c6ae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:50.598140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ea48719c-3691-4557-aab0-f44690c6ae2c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 15:59:50.599256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ea48719c-3691-4557-aab0-f44690c6ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 15:59:50.671281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 655/1303] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 15:59:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 15:59:50.947195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852acf7a-ca8a-4b1b-a40b-ac514f4875f2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:51.043024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:51.049737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852acf7a-ca8a-4b1b-a40b-ac514f4875f2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:51.084082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:51.084701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:51.084980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:51.085250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:51.085853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:51.087194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 15:59:51.096131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 15:59:51.102766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Sep 04 15:59:51.103656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 649/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Sep 4 15:59:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 15:59:51.104463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48719c-3691-4557-aab0-f44690c6ae2c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:51.182171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 15:59:51.204739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48719c-3691-4557-aab0-f44690c6ae2c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:51.237331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195d718-0bf1-4f89-ae8e-80093998ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:51.238001 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195d718-0bf1-4f89-ae8e-80093998ed3f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:51.238289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195d718-0bf1-4f89-ae8e-80093998ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:51.238559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195d718-0bf1-4f89-ae8e-80093998ed3f None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:51.239183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195d718-0bf1-4f89-ae8e-80093998ed3f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 15:59:51.239484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195d718-0bf1-4f89-ae8e-80093998ed3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:51.240845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195d718-0bf1-4f89-ae8e-80093998ed3f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:51.245873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 656/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Sep 4 15:59:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 15:59:51.251737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 15:59:51.350217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0b3e-715e-4d06-b2f3-be9dcb018209 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 15:59:51.383714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2cdd1-ba5b-41f8-adb8-69479a54f99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 15:59:51.384305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2cdd1-ba5b-41f8-adb8-69479a54f99d None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 15:59:51.384582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2cdd1-ba5b-41f8-adb8-69479a54f99d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 15:59:51.384852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2cdd1-ba5b-41f8-adb8-69479a54f99d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 15:59:51.385513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2cdd1-ba5b-41f8-adb8-69479a54f99d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 15:59:51.386981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2cdd1-ba5b-41f8-adb8-69479a54f99d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 15:59:51.479097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 650/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 15:59:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:00:01.295265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=30 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:00:01.474513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:38.807322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95060075-8873-4e43-82b8-1f6085e610f0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:38.811813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95060075-8873-4e43-82b8-1f6085e610f0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:38.811813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95060075-8873-4e43-82b8-1f6085e610f0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:38.814876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46b1de-09f2-41b8-964f-e5f8d355ecd3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:38.815331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46b1de-09f2-41b8-964f-e5f8d355ecd3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:38.815411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46b1de-09f2-41b8-964f-e5f8d355ecd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:38.820400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 657/1307] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:05:38 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:38.970709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ed46b1de-09f2-41b8-964f-e5f8d355ecd3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:38.973392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ed46b1de-09f2-41b8-964f-e5f8d355ecd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:39.097810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 651/1308] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:05:39.402709 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658da81-07da-4cff-a762-a700ed301e69 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:39.402997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658da81-07da-4cff-a762-a700ed301e69 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:39.403525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658da81-07da-4cff-a762-a700ed301e69 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:39.404975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 658/1309] 10.222.0.39 () {54 vars in 965 bytes} [Sun Sep 4 16:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:39.411502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbe10f-6e3e-44e0-9e28-1836c86f7c2c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:39.412089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbe10f-6e3e-44e0-9e28-1836c86f7c2c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:39.412440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbe10f-6e3e-44e0-9e28-1836c86f7c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:39.503457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0dfbe10f-6e3e-44e0-9e28-1836c86f7c2c None None] MFA Rules not processed for user `93ef027007184b8180325d6fb26324a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:39.505289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0dfbe10f-6e3e-44e0-9e28-1836c86f7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:39.529896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:39.589230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 652/1310] 10.222.0.39 () {58 vars in 1062 bytes} [Sun Sep 4 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:05:39.598206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658da81-07da-4cff-a762-a700ed301e69 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:05:39.600054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e658da81-07da-4cff-a762-a700ed301e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:39.715123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658da81-07da-4cff-a762-a700ed301e69 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:05:39.792735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a01adf-5d88-40ba-9316-2cb9e2fa9d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:05:39.793263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a01adf-5d88-40ba-9316-2cb9e2fa9d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:39.793547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a01adf-5d88-40ba-9316-2cb9e2fa9d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:39.793845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a01adf-5d88-40ba-9316-2cb9e2fa9d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:39.795437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c0a01adf-5d88-40ba-9316-2cb9e2fa9d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:39.851688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:39.910679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a01adf-5d88-40ba-9316-2cb9e2fa9d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:05:39.912206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a01adf-5d88-40ba-9316-2cb9e2fa9d6e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:05:39.931892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 659/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:05:42.175241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f215b4-6f92-401c-8178-d944b6457ab7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:42.175697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f215b4-6f92-401c-8178-d944b6457ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:42.175985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f215b4-6f92-401c-8178-d944b6457ab7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:42.177333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 653/1312] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:05:42.182268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fa12-dca4-476e-9ceb-2af162e19911 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:42.182645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fa12-dca4-476e-9ceb-2af162e19911 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:42.182823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fa12-dca4-476e-9ceb-2af162e19911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:42.257384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-86b4fa12-dca4-476e-9ceb-2af162e19911 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:42.258642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-86b4fa12-dca4-476e-9ceb-2af162e19911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:42.357981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 660/1313] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:05:42.663500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a560-b81c-4ca4-9dd6-367db4749968 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:42.663789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a560-b81c-4ca4-9dd6-367db4749968 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:42.664069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a560-b81c-4ca4-9dd6-367db4749968 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:42.665406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 654/1314] 10.222.0.39 () {54 vars in 965 bytes} [Sun Sep 4 16:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:42.671316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0314dac-995b-48c9-8f9d-bc44fe9dedc8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:42.671604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0314dac-995b-48c9-8f9d-bc44fe9dedc8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:42.671883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0314dac-995b-48c9-8f9d-bc44fe9dedc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:42.684054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:42.754730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c0314dac-995b-48c9-8f9d-bc44fe9dedc8 None None] MFA Rules not processed for user `93ef027007184b8180325d6fb26324a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:42.756196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c0314dac-995b-48c9-8f9d-bc44fe9dedc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:42.837860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 661/1315] 10.222.0.39 () {58 vars in 1062 bytes} [Sun Sep 4 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:05:42.846723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0a560-b81c-4ca4-9dd6-367db4749968 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:05:42.848941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eaa0a560-b81c-4ca4-9dd6-367db4749968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:42.952376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0a560-b81c-4ca4-9dd6-367db4749968 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:05:43.023190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:43.025905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafce6f9-5171-4715-ab42-bff8755e91ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:05:43.026520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce6f9-5171-4715-ab42-bff8755e91ec service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:43.026800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce6f9-5171-4715-ab42-bff8755e91ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:43.027071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce6f9-5171-4715-ab42-bff8755e91ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:43.028675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bafce6f9-5171-4715-ab42-bff8755e91ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:43.159879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafce6f9-5171-4715-ab42-bff8755e91ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:05:43.162254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafce6f9-5171-4715-ab42-bff8755e91ec service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:05:43.182723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 655/1316] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:05:45.792930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efc7b4-9b5a-4641-a5a1-5131154b5375 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:45.793438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efc7b4-9b5a-4641-a5a1-5131154b5375 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:45.793438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efc7b4-9b5a-4641-a5a1-5131154b5375 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:45.794785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 662/1317] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:45.800100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e358-a6dd-4440-bff7-191a91ec8f18 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:45.800645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e358-a6dd-4440-bff7-191a91ec8f18 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:45.800749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e358-a6dd-4440-bff7-191a91ec8f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:45.868144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a167e358-a6dd-4440-bff7-191a91ec8f18 None None] MFA Rules not processed for user `dce6be918e8741208b14eb3247cf7e38`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:45.869346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a167e358-a6dd-4440-bff7-191a91ec8f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:45.923134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:45.969047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 656/1318] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:05:46.261962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efc7b4-9b5a-4641-a5a1-5131154b5375 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:05:46.263495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-39efc7b4-9b5a-4641-a5a1-5131154b5375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:46.367053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efc7b4-9b5a-4641-a5a1-5131154b5375 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:05:46.369880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:46.441016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f76f2-07b3-4742-b009-d4009dd3f99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:05:46.442194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f76f2-07b3-4742-b009-d4009dd3f99b service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:46.442472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f76f2-07b3-4742-b009-d4009dd3f99b service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:46.442742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f76f2-07b3-4742-b009-d4009dd3f99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:46.444371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7f2f76f2-07b3-4742-b009-d4009dd3f99b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:46.553872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f76f2-07b3-4742-b009-d4009dd3f99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:05:46.555322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f76f2-07b3-4742-b009-d4009dd3f99b service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:05:46.573947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 663/1319] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:05:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:05:49.317625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b5d33-110d-4763-8086-4b231fa4f9a4 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:49.318194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b5d33-110d-4763-8086-4b231fa4f9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:49.318194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b5d33-110d-4763-8086-4b231fa4f9a4 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:49.319543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 657/1320] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:49.324744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577d5d5-171c-4c9c-a3f9-6ce831573672 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:49.325165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577d5d5-171c-4c9c-a3f9-6ce831573672 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:49.325294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577d5d5-171c-4c9c-a3f9-6ce831573672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:49.366313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:49.390365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3577d5d5-171c-4c9c-a3f9-6ce831573672 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:49.391555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3577d5d5-171c-4c9c-a3f9-6ce831573672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:49.492710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 664/1321] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:05:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:05:49.782735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b5d33-110d-4763-8086-4b231fa4f9a4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:05:49.784279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7f8b5d33-110d-4763-8086-4b231fa4f9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:49.894843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b5d33-110d-4763-8086-4b231fa4f9a4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:05:49.971195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b502c4-1c63-47fb-bdb9-26d49c6d638d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:05:49.971796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b502c4-1c63-47fb-bdb9-26d49c6d638d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:49.972072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b502c4-1c63-47fb-bdb9-26d49c6d638d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:49.972939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b502c4-1c63-47fb-bdb9-26d49c6d638d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:49.974781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-22b502c4-1c63-47fb-bdb9-26d49c6d638d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:49.982112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:50.079804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b502c4-1c63-47fb-bdb9-26d49c6d638d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:05:50.081150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b502c4-1c63-47fb-bdb9-26d49c6d638d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:05:50.099480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 658/1322] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:05:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:05:52.990473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d342e9e-f0f9-481e-8f5f-f6a6564eceaf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:52.991042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d342e9e-f0f9-481e-8f5f-f6a6564eceaf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:52.991042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d342e9e-f0f9-481e-8f5f-f6a6564eceaf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:52.992382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 665/1323] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:52.997606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc8d4-2f11-4cb2-bcc0-25fe35442a2e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:52.998034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc8d4-2f11-4cb2-bcc0-25fe35442a2e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:52.998167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc8d4-2f11-4cb2-bcc0-25fe35442a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:53.064185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-622dc8d4-2f11-4cb2-bcc0-25fe35442a2e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:53.065437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-622dc8d4-2f11-4cb2-bcc0-25fe35442a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:53.124360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:53.281044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 659/1324] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:05:53.564671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d342e9e-f0f9-481e-8f5f-f6a6564eceaf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:05:53.566285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2d342e9e-f0f9-481e-8f5f-f6a6564eceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:53.670790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d342e9e-f0f9-481e-8f5f-f6a6564eceaf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:05:53.683455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:53.745974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db3a46-4c1c-43ab-91ea-e72278a73e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:05:53.746505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db3a46-4c1c-43ab-91ea-e72278a73e9a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:53.746787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db3a46-4c1c-43ab-91ea-e72278a73e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:53.747057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db3a46-4c1c-43ab-91ea-e72278a73e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:53.748650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b7db3a46-4c1c-43ab-91ea-e72278a73e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:53.861832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db3a46-4c1c-43ab-91ea-e72278a73e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:05:53.863845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db3a46-4c1c-43ab-91ea-e72278a73e9a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:05:53.883061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 666/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:05:55.865465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1491c-2f9e-4641-8531-84cbd2aa5e0a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:55.866033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1491c-2f9e-4641-8531-84cbd2aa5e0a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:55.866033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1491c-2f9e-4641-8531-84cbd2aa5e0a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:55.867365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 660/1326] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:55.872488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19ffd4-e90d-4e76-a181-b5cd5cf26c01 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:55.872898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19ffd4-e90d-4e76-a181-b5cd5cf26c01 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:55.873141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19ffd4-e90d-4e76-a181-b5cd5cf26c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:55.922162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:55.939941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bb19ffd4-e90d-4e76-a181-b5cd5cf26c01 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:55.941321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bb19ffd4-e90d-4e76-a181-b5cd5cf26c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:56.039455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 667/1327] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:05:56.332403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1491c-2f9e-4641-8531-84cbd2aa5e0a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:05:56.334087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-33f1491c-2f9e-4641-8531-84cbd2aa5e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:56.463244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1491c-2f9e-4641-8531-84cbd2aa5e0a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:05:56.539175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5ae45-43f2-4bd3-9a67-b0528d78d51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:05:56.539777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5ae45-43f2-4bd3-9a67-b0528d78d51e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:56.540053 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5ae45-43f2-4bd3-9a67-b0528d78d51e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:56.540323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5ae45-43f2-4bd3-9a67-b0528d78d51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:56.541922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-74b5ae45-43f2-4bd3-9a67-b0528d78d51e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:56.558921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:56.654227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5ae45-43f2-4bd3-9a67-b0528d78d51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:05:56.655892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5ae45-43f2-4bd3-9a67-b0528d78d51e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:05:56.674392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 661/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:05:58.714485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164b2b0-755c-4277-8969-99a844a059ac None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:58.715014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164b2b0-755c-4277-8969-99a844a059ac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:58.715129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164b2b0-755c-4277-8969-99a844a059ac None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:58.716403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 668/1329] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:05:58.721677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a75b5-9037-464b-8417-c3e0e98a74e1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:58.722092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a75b5-9037-464b-8417-c3e0e98a74e1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:58.722805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a75b5-9037-464b-8417-c3e0e98a74e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:58.790091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2c0a75b5-9037-464b-8417-c3e0e98a74e1 None None] MFA Rules not processed for user `dce6be918e8741208b14eb3247cf7e38`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:05:58.791301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c0a75b5-9037-464b-8417-c3e0e98a74e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:58.860498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:58.893010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 662/1330] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:05:59.193397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164b2b0-755c-4277-8969-99a844a059ac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:05:59.195053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c164b2b0-755c-4277-8969-99a844a059ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:59.302826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164b2b0-755c-4277-8969-99a844a059ac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:05:59.329346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:05:59.399384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66467c58-0098-4a81-a3ea-9d5ab1ff394e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:05:59.399899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66467c58-0098-4a81-a3ea-9d5ab1ff394e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:05:59.400182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66467c58-0098-4a81-a3ea-9d5ab1ff394e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:05:59.400459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66467c58-0098-4a81-a3ea-9d5ab1ff394e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:05:59.402199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-66467c58-0098-4a81-a3ea-9d5ab1ff394e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:05:59.538188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66467c58-0098-4a81-a3ea-9d5ab1ff394e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:05:59.539660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66467c58-0098-4a81-a3ea-9d5ab1ff394e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:05:59.564298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 669/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:05:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:02.002324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd442ab-28b0-4be7-9907-70405b2629cb None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:02.002981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd442ab-28b0-4be7-9907-70405b2629cb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:02.002981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd442ab-28b0-4be7-9907-70405b2629cb None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:02.006991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 663/1332] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:02.013984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e054c2-c6c5-4573-8396-75c6e7cd1c5f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:02.014463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e054c2-c6c5-4573-8396-75c6e7cd1c5f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:02.014564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e054c2-c6c5-4573-8396-75c6e7cd1c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:02.079356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:02.082299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-41e054c2-c6c5-4573-8396-75c6e7cd1c5f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:02.083501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-41e054c2-c6c5-4573-8396-75c6e7cd1c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:02.178999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 670/1333] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:02.476306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd442ab-28b0-4be7-9907-70405b2629cb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:02.477924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1dd442ab-28b0-4be7-9907-70405b2629cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:02.579181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd442ab-28b0-4be7-9907-70405b2629cb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:02.652052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c52dba-ce7e-41e3-85a7-01df8a51511e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:02.652659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c52dba-ce7e-41e3-85a7-01df8a51511e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:02.652940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c52dba-ce7e-41e3-85a7-01df8a51511e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:02.653209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c52dba-ce7e-41e3-85a7-01df8a51511e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:02.654836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-86c52dba-ce7e-41e3-85a7-01df8a51511e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:02.682447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:02.763709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c52dba-ce7e-41e3-85a7-01df8a51511e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:02.765139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c52dba-ce7e-41e3-85a7-01df8a51511e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:02.783768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 664/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:04.799903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e76bb-ff30-4833-9759-72fda2383092 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:04.801004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e76bb-ff30-4833-9759-72fda2383092 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:04.801585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e76bb-ff30-4833-9759-72fda2383092 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:04.802968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 671/1335] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:04.808231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d64c-7938-42cc-876c-18c4397a5c1d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:04.808609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d64c-7938-42cc-876c-18c4397a5c1d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:04.808784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d64c-7938-42cc-876c-18c4397a5c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:04.873719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a7c4d64c-7938-42cc-876c-18c4397a5c1d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:04.874910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a7c4d64c-7938-42cc-876c-18c4397a5c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:04.950680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:04.971465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 665/1336] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:05.256758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e76bb-ff30-4833-9759-72fda2383092 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:05.259576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c02e76bb-ff30-4833-9759-72fda2383092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:05.366571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e76bb-ff30-4833-9759-72fda2383092 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:05.395831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:05.439650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02406965-860e-49fe-ad80-a7ee57a694af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:05.440278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02406965-860e-49fe-ad80-a7ee57a694af service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:05.440555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02406965-860e-49fe-ad80-a7ee57a694af service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:05.441379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02406965-860e-49fe-ad80-a7ee57a694af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:05.443255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-02406965-860e-49fe-ad80-a7ee57a694af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:05.553630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02406965-860e-49fe-ad80-a7ee57a694af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:05.555016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02406965-860e-49fe-ad80-a7ee57a694af service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:05.578640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 672/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:08.422542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8049c58-32fc-4fbe-ada3-b2e30cb0094e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:08.423161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8049c58-32fc-4fbe-ada3-b2e30cb0094e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:08.423161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8049c58-32fc-4fbe-ada3-b2e30cb0094e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:08.424454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 666/1338] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:08.430496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1884a-12df-4b1c-8dfd-e52f3d57e161 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:08.430933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1884a-12df-4b1c-8dfd-e52f3d57e161 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:08.431035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1884a-12df-4b1c-8dfd-e52f3d57e161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:08.498154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-74f1884a-12df-4b1c-8dfd-e52f3d57e161 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:08.499382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74f1884a-12df-4b1c-8dfd-e52f3d57e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:08.508200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:08.597971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 673/1339] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:08.889287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8049c58-32fc-4fbe-ada3-b2e30cb0094e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:08.890841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a8049c58-32fc-4fbe-ada3-b2e30cb0094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:08.994525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8049c58-32fc-4fbe-ada3-b2e30cb0094e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:09.069891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51c93f-2a6f-4551-adfa-f449293c3f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:09.070507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c93f-2a6f-4551-adfa-f449293c3f52 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:09.070785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c93f-2a6f-4551-adfa-f449293c3f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:09.071054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c93f-2a6f-4551-adfa-f449293c3f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:09.072731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cd51c93f-2a6f-4551-adfa-f449293c3f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:09.109132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:09.182460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51c93f-2a6f-4551-adfa-f449293c3f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:09.183814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51c93f-2a6f-4551-adfa-f449293c3f52 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:09.201964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 667/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:13.225338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25803bf-805f-4f43-b07d-d6180897f630 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:13.225901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25803bf-805f-4f43-b07d-d6180897f630 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:13.225901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25803bf-805f-4f43-b07d-d6180897f630 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:13.227333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 674/1341] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:13.232387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fca20-0af0-4f03-b6f9-b0c17af18ec5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:13.232801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fca20-0af0-4f03-b6f9-b0c17af18ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:13.232942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fca20-0af0-4f03-b6f9-b0c17af18ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:13.305752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-403fca20-0af0-4f03-b6f9-b0c17af18ec5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:13.306972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-403fca20-0af0-4f03-b6f9-b0c17af18ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:13.397989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:13.409419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 668/1342] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:13.691722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25803bf-805f-4f43-b07d-d6180897f630 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:13.693266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e25803bf-805f-4f43-b07d-d6180897f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:13.796593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25803bf-805f-4f43-b07d-d6180897f630 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:13.834734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:13.874382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4b01a-365b-4b6b-8ab7-51a3952497c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:13.874992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b01a-365b-4b6b-8ab7-51a3952497c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:13.875270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b01a-365b-4b6b-8ab7-51a3952497c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:13.875541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b01a-365b-4b6b-8ab7-51a3952497c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:13.877145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0ee4b01a-365b-4b6b-8ab7-51a3952497c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:13.985241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4b01a-365b-4b6b-8ab7-51a3952497c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:13.986631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4b01a-365b-4b6b-8ab7-51a3952497c4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:14.007099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 675/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:16.797199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1fda-8ab4-4c6b-acc2-f800331aca49 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:16.797752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1fda-8ab4-4c6b-acc2-f800331aca49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:16.797879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1fda-8ab4-4c6b-acc2-f800331aca49 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:16.799165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 669/1344] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:16.805380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf54cd-1363-4754-9448-279dbff679fc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:16.805815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf54cd-1363-4754-9448-279dbff679fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:16.805972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf54cd-1363-4754-9448-279dbff679fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:16.874740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-edcf54cd-1363-4754-9448-279dbff679fc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:16.875944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-edcf54cd-1363-4754-9448-279dbff679fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:16.894944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:16.973934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 676/1345] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:17.260494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad1fda-8ab4-4c6b-acc2-f800331aca49 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:17.262055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-adad1fda-8ab4-4c6b-acc2-f800331aca49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:17.369644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad1fda-8ab4-4c6b-acc2-f800331aca49 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:17.447999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747d398-b972-4c67-85bf-866888923097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:17.448618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747d398-b972-4c67-85bf-866888923097 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:17.448900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747d398-b972-4c67-85bf-866888923097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:17.449308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747d398-b972-4c67-85bf-866888923097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:17.451012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c747d398-b972-4c67-85bf-866888923097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:17.496427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:17.569659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747d398-b972-4c67-85bf-866888923097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:17.571047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747d398-b972-4c67-85bf-866888923097 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:17.591274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 670/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:21.855871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b881a2-5969-4685-b880-3a0a522cb86d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:21.856363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b881a2-5969-4685-b880-3a0a522cb86d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:21.856465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b881a2-5969-4685-b880-3a0a522cb86d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:21.857842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 677/1347] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:21.864202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f6e63-d89a-48dc-8b21-c7772c57cbba None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:21.864716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f6e63-d89a-48dc-8b21-c7772c57cbba None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:21.864814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f6e63-d89a-48dc-8b21-c7772c57cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:21.935778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-be3f6e63-d89a-48dc-8b21-c7772c57cbba None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:21.937042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-be3f6e63-d89a-48dc-8b21-c7772c57cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:22.039428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:22.042043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 671/1348] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:22.332797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b881a2-5969-4685-b880-3a0a522cb86d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:22.334928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-28b881a2-5969-4685-b880-3a0a522cb86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:22.443642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b881a2-5969-4685-b880-3a0a522cb86d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:22.492442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:22.520439 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791622f1-b75a-4fa6-97ac-b4b68495bd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:22.521096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791622f1-b75a-4fa6-97ac-b4b68495bd32 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:22.521341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791622f1-b75a-4fa6-97ac-b4b68495bd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:22.521632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791622f1-b75a-4fa6-97ac-b4b68495bd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:22.523234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-791622f1-b75a-4fa6-97ac-b4b68495bd32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:22.630362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791622f1-b75a-4fa6-97ac-b4b68495bd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:22.632075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791622f1-b75a-4fa6-97ac-b4b68495bd32 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:22.650469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 678/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:27.611987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8206b4-3874-4337-8eda-92804c71c4cb None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:27.613038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8206b4-3874-4337-8eda-92804c71c4cb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:27.613271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8206b4-3874-4337-8eda-92804c71c4cb None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:27.630012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad49141-40f9-49df-b3b6-6debf95b84dd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:27.630012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad49141-40f9-49df-b3b6-6debf95b84dd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:27.630012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad49141-40f9-49df-b3b6-6debf95b84dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:27.630893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 672/1350] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:06:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:27.718474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0ad49141-40f9-49df-b3b6-6debf95b84dd None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:27.719769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0ad49141-40f9-49df-b3b6-6debf95b84dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:27.772369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:27.884468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 679/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:06:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:31.987510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff36f7b-56c8-4eb9-9727-b81cc803dc66 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:31.988033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff36f7b-56c8-4eb9-9727-b81cc803dc66 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:31.988150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff36f7b-56c8-4eb9-9727-b81cc803dc66 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:31.989685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 673/1352] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:31.995099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d976d-6293-4bc4-b235-f5b45dd079e0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:31.995541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d976d-6293-4bc4-b235-f5b45dd079e0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:31.995782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d976d-6293-4bc4-b235-f5b45dd079e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:32.079940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d52d976d-6293-4bc4-b235-f5b45dd079e0 None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:32.081370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d52d976d-6293-4bc4-b235-f5b45dd079e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:32.164979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:32.207382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 680/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:33.704991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88babcb-c79e-418d-a8db-8b628c370e12 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:33.705524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88babcb-c79e-418d-a8db-8b628c370e12 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:33.705524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88babcb-c79e-418d-a8db-8b628c370e12 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:33.707023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 674/1354] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:33.712539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4612c6-15d0-4931-b92a-0c2fb65f41b0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:33.713008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4612c6-15d0-4931-b92a-0c2fb65f41b0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:33.713113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4612c6-15d0-4931-b92a-0c2fb65f41b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:33.789549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2f4612c6-15d0-4931-b92a-0c2fb65f41b0 None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:33.790774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2f4612c6-15d0-4931-b92a-0c2fb65f41b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:33.897262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 681/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:37.312164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123aed4-422f-412f-81d8-63559a7e4a9e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:37.312721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123aed4-422f-412f-81d8-63559a7e4a9e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:37.312721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123aed4-422f-412f-81d8-63559a7e4a9e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:37.314122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 675/1356] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:37.319479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cdecf-ba1f-4571-ab3c-119cb8fbe4e0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:37.320273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cdecf-ba1f-4571-ab3c-119cb8fbe4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:37.320761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cdecf-ba1f-4571-ab3c-119cb8fbe4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:37.359531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:37.403372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-417cdecf-ba1f-4571-ab3c-119cb8fbe4e0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:37.404856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-417cdecf-ba1f-4571-ab3c-119cb8fbe4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:37.537666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 682/1357] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:37.852818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6615c-1702-421f-8c14-01442ba7410b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:37.853386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6615c-1702-421f-8c14-01442ba7410b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:37.853990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6615c-1702-421f-8c14-01442ba7410b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:37.855586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 676/1358] 10.222.0.39 () {54 vars in 962 bytes} [Sun Sep 4 16:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:37.876485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65232ac-6bae-49ca-9c86-14076a2b2ac1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:37.876918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65232ac-6bae-49ca-9c86-14076a2b2ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:37.877298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65232ac-6bae-49ca-9c86-14076a2b2ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:37.969335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f65232ac-6bae-49ca-9c86-14076a2b2ac1 None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:37.971204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f65232ac-6bae-49ca-9c86-14076a2b2ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:38.026599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:38.095627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 683/1359] 10.222.0.39 () {58 vars in 1059 bytes} [Sun Sep 4 16:06:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:38.103499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6615c-1702-421f-8c14-01442ba7410b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:38.106409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5fc6615c-1702-421f-8c14-01442ba7410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:38.277144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6615c-1702-421f-8c14-01442ba7410b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:38.357469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:38.401440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b0ce9-dab5-4b3e-ad9a-dca2dbb9ddf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:38.402072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b0ce9-dab5-4b3e-ad9a-dca2dbb9ddf3 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:38.402351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b0ce9-dab5-4b3e-ad9a-dca2dbb9ddf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:38.402617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b0ce9-dab5-4b3e-ad9a-dca2dbb9ddf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:38.404188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ba3b0ce9-dab5-4b3e-ad9a-dca2dbb9ddf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:38.496709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b0ce9-dab5-4b3e-ad9a-dca2dbb9ddf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:38.498319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b0ce9-dab5-4b3e-ad9a-dca2dbb9ddf3 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:38.517544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 677/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:40.676891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb3322-b8c5-44cd-9177-5834a68a1c8b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:40.677402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb3322-b8c5-44cd-9177-5834a68a1c8b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:40.677500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb3322-b8c5-44cd-9177-5834a68a1c8b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:40.678903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 684/1361] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:06:40.684858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11782b-46a7-44db-b566-090ef771bf73 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:40.687419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11782b-46a7-44db-b566-090ef771bf73 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:40.687419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11782b-46a7-44db-b566-090ef771bf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:40.757651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2f11782b-46a7-44db-b566-090ef771bf73 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:40.758851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f11782b-46a7-44db-b566-090ef771bf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:40.798138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:40.865741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 678/1362] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:41.176398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb3322-b8c5-44cd-9177-5834a68a1c8b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:41.180986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59eb3322-b8c5-44cd-9177-5834a68a1c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:41.338370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb3322-b8c5-44cd-9177-5834a68a1c8b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:41.348479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:41.458359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18582e4c-dacb-4063-9cc0-9b3a2b6f3cdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:41.459128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18582e4c-dacb-4063-9cc0-9b3a2b6f3cdb service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:41.459509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18582e4c-dacb-4063-9cc0-9b3a2b6f3cdb service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:41.459887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18582e4c-dacb-4063-9cc0-9b3a2b6f3cdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:41.461680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-18582e4c-dacb-4063-9cc0-9b3a2b6f3cdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:41.565909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18582e4c-dacb-4063-9cc0-9b3a2b6f3cdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:41.567527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18582e4c-dacb-4063-9cc0-9b3a2b6f3cdb service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:41.587842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 685/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:43.982830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f23a2e-8bc2-43c2-8148-30427c5f2e19 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:43.983713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f23a2e-8bc2-43c2-8148-30427c5f2e19 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:43.983994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f23a2e-8bc2-43c2-8148-30427c5f2e19 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:43.985339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 679/1364] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:43.991687 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0f76e-45a2-43a9-98b5-e80c4ddc17ee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:43.992539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0f76e-45a2-43a9-98b5-e80c4ddc17ee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:43.992983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0f76e-45a2-43a9-98b5-e80c4ddc17ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:44.010697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:44.064545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6ae0f76e-45a2-43a9-98b5-e80c4ddc17ee None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:44.067019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6ae0f76e-45a2-43a9-98b5-e80c4ddc17ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:44.185644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 686/1365] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:44.477771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1267c2-0f0f-4c47-afb8-f396ae94a0a7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:44.478289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1267c2-0f0f-4c47-afb8-f396ae94a0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:44.478764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1267c2-0f0f-4c47-afb8-f396ae94a0a7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:44.480385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 680/1366] 10.222.0.39 () {54 vars in 962 bytes} [Sun Sep 4 16:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:44.485804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002650c6-2a59-4877-a2ff-8a3b4f5468b1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:44.486103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002650c6-2a59-4877-a2ff-8a3b4f5468b1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:44.486390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002650c6-2a59-4877-a2ff-8a3b4f5468b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:44.576041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-002650c6-2a59-4877-a2ff-8a3b4f5468b1 None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:44.577299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-002650c6-2a59-4877-a2ff-8a3b4f5468b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:44.616387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:44.709177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 687/1367] 10.222.0.39 () {58 vars in 1059 bytes} [Sun Sep 4 16:06:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:44.717987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1267c2-0f0f-4c47-afb8-f396ae94a0a7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:44.719677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6c1267c2-0f0f-4c47-afb8-f396ae94a0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:44.878061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1267c2-0f0f-4c47-afb8-f396ae94a0a7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:44.900817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:45.003632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1e77-eb45-49e5-a39d-6ff42fea3d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:45.004400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1e77-eb45-49e5-a39d-6ff42fea3d43 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:45.004773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1e77-eb45-49e5-a39d-6ff42fea3d43 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:45.005135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1e77-eb45-49e5-a39d-6ff42fea3d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:45.007104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3a0b1e77-eb45-49e5-a39d-6ff42fea3d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:45.103650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b1e77-eb45-49e5-a39d-6ff42fea3d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:45.105380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b1e77-eb45-49e5-a39d-6ff42fea3d43 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:45.123707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 681/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:45.661578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a3b1-4bb6-4e24-977d-997cf75ac700 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:45.662143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a3b1-4bb6-4e24-977d-997cf75ac700 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:45.662143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a3b1-4bb6-4e24-977d-997cf75ac700 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:45.663485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 688/1369] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:45.668837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3c67-51be-4ae1-b0fe-0488715bcd1d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:45.669234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3c67-51be-4ae1-b0fe-0488715bcd1d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:45.669780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3c67-51be-4ae1-b0fe-0488715bcd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:45.702837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:45.739163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5dbf3c67-51be-4ae1-b0fe-0488715bcd1d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:45.740506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5dbf3c67-51be-4ae1-b0fe-0488715bcd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:45.841890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 682/1370] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:46.424342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd41d3-bca0-4c96-8278-d6b70f31315d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:46.424342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd41d3-bca0-4c96-8278-d6b70f31315d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:46.424342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd41d3-bca0-4c96-8278-d6b70f31315d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:46.425730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 689/1371] 10.222.0.39 () {54 vars in 964 bytes} [Sun Sep 4 16:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:46.430540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633a74f-a366-4b15-8e0f-711e9f38e5d9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:46.431127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633a74f-a366-4b15-8e0f-711e9f38e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:46.431127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633a74f-a366-4b15-8e0f-711e9f38e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:46.511302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6633a74f-a366-4b15-8e0f-711e9f38e5d9 None None] MFA Rules not processed for user `c029d59023b04bf8b8bf9d4d5eee7390`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:46.512511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6633a74f-a366-4b15-8e0f-711e9f38e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:46.561786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:46.590902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 683/1372] 10.222.0.39 () {58 vars in 1061 bytes} [Sun Sep 4 16:06:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:46.599059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd41d3-bca0-4c96-8278-d6b70f31315d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:46.600815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-80bd41d3-bca0-4c96-8278-d6b70f31315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:46.711250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd41d3-bca0-4c96-8278-d6b70f31315d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:46.754399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:46.790802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e95c2-8059-4f90-8f15-78de8c770c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:46.791412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e95c2-8059-4f90-8f15-78de8c770c74 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:46.791691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e95c2-8059-4f90-8f15-78de8c770c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:46.791962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e95c2-8059-4f90-8f15-78de8c770c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:46.793653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dc0e95c2-8059-4f90-8f15-78de8c770c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:46.916781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e95c2-8059-4f90-8f15-78de8c770c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:46.918297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e95c2-8059-4f90-8f15-78de8c770c74 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:46.938604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 690/1373] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:47.290446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5e370-98c9-4652-8a19-cfbd21757c36 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:47.290735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5e370-98c9-4652-8a19-cfbd21757c36 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:47.291014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5e370-98c9-4652-8a19-cfbd21757c36 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:47.292368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 684/1374] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:47.297669 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309653d3-ee2d-4b2b-a72c-8c3190426c59 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:47.297959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309653d3-ee2d-4b2b-a72c-8c3190426c59 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:47.298236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309653d3-ee2d-4b2b-a72c-8c3190426c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:47.367664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-309653d3-ee2d-4b2b-a72c-8c3190426c59 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:47.368858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-309653d3-ee2d-4b2b-a72c-8c3190426c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:47.389880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:47.467322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 691/1375] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:47.760558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5e370-98c9-4652-8a19-cfbd21757c36 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:47.762129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c9b5e370-98c9-4652-8a19-cfbd21757c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:47.912642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5e370-98c9-4652-8a19-cfbd21757c36 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:47.963586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:48.024942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d096e6-1880-41f4-b2f7-dd52009f98bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:48.025551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d096e6-1880-41f4-b2f7-dd52009f98bb service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:48.025864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d096e6-1880-41f4-b2f7-dd52009f98bb service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:48.026146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d096e6-1880-41f4-b2f7-dd52009f98bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:48.027745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-88d096e6-1880-41f4-b2f7-dd52009f98bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:48.117865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d096e6-1880-41f4-b2f7-dd52009f98bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:48.119452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d096e6-1880-41f4-b2f7-dd52009f98bb service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:48.137841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 685/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:50.230525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c54a0-5e80-43f0-9146-ce7eb50b4607 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:50.231026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c54a0-5e80-43f0-9146-ce7eb50b4607 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:50.231026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c54a0-5e80-43f0-9146-ce7eb50b4607 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:50.232338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 692/1377] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:06:50.238920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456dd1b-e256-40b9-917b-83867e73efbf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:50.239696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456dd1b-e256-40b9-917b-83867e73efbf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:50.239974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456dd1b-e256-40b9-917b-83867e73efbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:50.312106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7456dd1b-e256-40b9-917b-83867e73efbf None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:50.313306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7456dd1b-e256-40b9-917b-83867e73efbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:50.320619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:50.412115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 686/1378] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:50.733073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c54a0-5e80-43f0-9146-ce7eb50b4607 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:50.734363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-150c54a0-5e80-43f0-9146-ce7eb50b4607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:50.905408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c54a0-5e80-43f0-9146-ce7eb50b4607 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:50.911654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:51.036438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22849c78-6eef-4a12-b6d6-832024864674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:51.036970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22849c78-6eef-4a12-b6d6-832024864674 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:51.037254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22849c78-6eef-4a12-b6d6-832024864674 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:51.037544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22849c78-6eef-4a12-b6d6-832024864674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:51.039188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-22849c78-6eef-4a12-b6d6-832024864674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:51.135079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22849c78-6eef-4a12-b6d6-832024864674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:51.136689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22849c78-6eef-4a12-b6d6-832024864674 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:51.156346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 693/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:53.263849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cbc08-0b39-4abc-8d74-90d6bc803e66 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:53.264367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cbc08-0b39-4abc-8d74-90d6bc803e66 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:53.264367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cbc08-0b39-4abc-8d74-90d6bc803e66 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:53.265819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 687/1380] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:53.271069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1732f-6f24-435f-971d-9d4cfef3d1e5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:53.271503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1732f-6f24-435f-971d-9d4cfef3d1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:53.271723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1732f-6f24-435f-971d-9d4cfef3d1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:53.282960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:53.347102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1be1732f-6f24-435f-971d-9d4cfef3d1e5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:53.348303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1be1732f-6f24-435f-971d-9d4cfef3d1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:53.452080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 694/1381] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:53.741136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cbc08-0b39-4abc-8d74-90d6bc803e66 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:53.743101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d97cbc08-0b39-4abc-8d74-90d6bc803e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:53.902509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:53.915015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cbc08-0b39-4abc-8d74-90d6bc803e66 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:54.044644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f71b0-04c9-4bda-8809-dd9c0088c3da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:54.045189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f71b0-04c9-4bda-8809-dd9c0088c3da service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:54.045466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f71b0-04c9-4bda-8809-dd9c0088c3da service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:54.045766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f71b0-04c9-4bda-8809-dd9c0088c3da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:54.047426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-978f71b0-04c9-4bda-8809-dd9c0088c3da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:54.146491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f71b0-04c9-4bda-8809-dd9c0088c3da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:54.148070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f71b0-04c9-4bda-8809-dd9c0088c3da service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:54.160126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:54.170595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 688/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:06:56.372668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f8a4-d47a-4edf-8759-a296d30e777b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:56.373178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f8a4-d47a-4edf-8759-a296d30e777b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:56.373494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f8a4-d47a-4edf-8759-a296d30e777b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:56.375461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 695/1383] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:56.381626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb6c6e-2423-4cd1-a8a1-1917f9ba8296 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:56.382352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb6c6e-2423-4cd1-a8a1-1917f9ba8296 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:56.382462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb6c6e-2423-4cd1-a8a1-1917f9ba8296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:56.454021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-32bb6c6e-2423-4cd1-a8a1-1917f9ba8296 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:56.455387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-32bb6c6e-2423-4cd1-a8a1-1917f9ba8296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:56.579713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 689/1384] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:56.876138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114f8a4-d47a-4edf-8759-a296d30e777b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:56.878397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0114f8a4-d47a-4edf-8759-a296d30e777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:56.966411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:57.039167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114f8a4-d47a-4edf-8759-a296d30e777b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:06:57.170347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b159a6-53de-4c8b-b8b8-eab3cd4f0de3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:06:57.171005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b159a6-53de-4c8b-b8b8-eab3cd4f0de3 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:57.171361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b159a6-53de-4c8b-b8b8-eab3cd4f0de3 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:57.171710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b159a6-53de-4c8b-b8b8-eab3cd4f0de3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:57.173785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f6b159a6-53de-4c8b-b8b8-eab3cd4f0de3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:57.240940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:57.297784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b159a6-53de-4c8b-b8b8-eab3cd4f0de3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:06:57.299954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b159a6-53de-4c8b-b8b8-eab3cd4f0de3 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:06:57.321720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 696/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:06:59.283574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc661835-61e3-4947-b89e-19662bcb5929 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:59.284217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc661835-61e3-4947-b89e-19662bcb5929 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:59.284217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc661835-61e3-4947-b89e-19662bcb5929 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:59.285551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 690/1386] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:06:59.291198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fdd97-05e4-489c-a6fe-ef055ad8e44e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:06:59.291592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fdd97-05e4-489c-a6fe-ef055ad8e44e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:06:59.291905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fdd97-05e4-489c-a6fe-ef055ad8e44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:06:59.386031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a41fdd97-05e4-489c-a6fe-ef055ad8e44e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:06:59.387624 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a41fdd97-05e4-489c-a6fe-ef055ad8e44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:59.500017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 697/1387] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:06:59.796548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc661835-61e3-4947-b89e-19662bcb5929 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:06:59.798203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dc661835-61e3-4947-b89e-19662bcb5929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:06:59.893425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:06:59.968129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc661835-61e3-4947-b89e-19662bcb5929 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:00.100073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3c50a-5453-4c7e-8817-91e312b950f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:00.100808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3c50a-5453-4c7e-8817-91e312b950f5 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:00.101106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3c50a-5453-4c7e-8817-91e312b950f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:00.101499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3c50a-5453-4c7e-8817-91e312b950f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:00.103245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b6d3c50a-5453-4c7e-8817-91e312b950f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:00.147982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:00.215879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3c50a-5453-4c7e-8817-91e312b950f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:00.217555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3c50a-5453-4c7e-8817-91e312b950f5 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:00.236495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 691/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:02.445620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01809cef-5505-4e33-a2ea-864c17d83f38 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:02.445620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01809cef-5505-4e33-a2ea-864c17d83f38 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:02.445620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01809cef-5505-4e33-a2ea-864c17d83f38 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:02.446385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 698/1389] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:02.451846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:02.452284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:02.452465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:02.458414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ee21b-8a15-4494-b8bd-f932f94b3792 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:02.458702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ee21b-8a15-4494-b8bd-f932f94b3792 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:02.459007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ee21b-8a15-4494-b8bd-f932f94b3792 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:02.460332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 699/1390] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:02.465427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474eb4-5e90-43e6-a8fd-2cb518dcdf4a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:02.465755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474eb4-5e90-43e6-a8fd-2cb518dcdf4a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:02.466031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474eb4-5e90-43e6-a8fd-2cb518dcdf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:02.507273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:02.522898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:02.524123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:02.543479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-04474eb4-5e90-43e6-a8fd-2cb518dcdf4a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:02.544776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-04474eb4-5e90-43e6-a8fd-2cb518dcdf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:02.624113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 692/1391] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:02.643510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 700/1392] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:07:02.918815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:02.919951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:02.934875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:02.946652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6a88-2502-4fc6-99de-6291805836c3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:02.947007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6a88-2502-4fc6-99de-6291805836c3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:02.947292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6a88-2502-4fc6-99de-6291805836c3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:02.949026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 701/1393] 10.222.0.39 () {54 vars in 961 bytes} [Sun Sep 4 16:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:07:02.954037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9ae78-9bac-4c55-92e9-a6f328ad4d49 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:02.954325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9ae78-9bac-4c55-92e9-a6f328ad4d49 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:02.954613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9ae78-9bac-4c55-92e9-a6f328ad4d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:03.033962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bdf9ae78-9bac-4c55-92e9-a6f328ad4d49 None None] MFA Rules not processed for user `c95204bd338a4a99a510a67c4ab97532`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:03.035167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bdf9ae78-9bac-4c55-92e9-a6f328ad4d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:03.042170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f74ec-76e1-490a-99f0-84deeca0e883 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:03.082501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:03.118298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 702/1394] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Sep 4 16:07:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:07:03.125852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9ae78-9bac-4c55-92e9-a6f328ad4d49 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:03.134457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9ae78-9bac-4c55-92e9-a6f328ad4d49 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:03.161638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c49f0-0fd3-4e91-bf9f-9ac47ace3fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:03.162182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c49f0-0fd3-4e91-bf9f-9ac47ace3fb2 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:03.162460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c49f0-0fd3-4e91-bf9f-9ac47ace3fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:03.162724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c49f0-0fd3-4e91-bf9f-9ac47ace3fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:03.164476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-356c49f0-0fd3-4e91-bf9f-9ac47ace3fb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:03.173639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:03.210268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f310f80-3b46-4a28-9be0-faad5254b4d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95204bd338a4a99a510a67c4ab97532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:03.210884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f310f80-3b46-4a28-9be0-faad5254b4d9 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:03.211192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f310f80-3b46-4a28-9be0-faad5254b4d9 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:03.211466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f310f80-3b46-4a28-9be0-faad5254b4d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:03.213084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f310f80-3b46-4a28-9be0-faad5254b4d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:03.258597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c49f0-0fd3-4e91-bf9f-9ac47ace3fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:03.260186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c49f0-0fd3-4e91-bf9f-9ac47ace3fb2 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:03.281103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 693/1395] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:03.334403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f310f80-3b46-4a28-9be0-faad5254b4d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:03.336604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f310f80-3b46-4a28-9be0-faad5254b4d9 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:03.350426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:03.355087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 703/1396] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:07:04.316347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ae7e3-9769-439c-a196-8451119b8159 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:04.316924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ae7e3-9769-439c-a196-8451119b8159 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:04.316924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ae7e3-9769-439c-a196-8451119b8159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:04.392575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-391ae7e3-9769-439c-a196-8451119b8159 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:04.394028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-391ae7e3-9769-439c-a196-8451119b8159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:04.479568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:04.482567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 694/1397] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:07:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:04.528506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd70d-65fb-41b7-97ed-1a367a50d549 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:04.528989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd70d-65fb-41b7-97ed-1a367a50d549 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:04.529155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd70d-65fb-41b7-97ed-1a367a50d549 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:04.530765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 704/1398] 10.222.0.39 () {54 vars in 943 bytes} [Sun Sep 4 16:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:04.538090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3e19-485f-43d0-9522-5fc1e2dd90cd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:04.538383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3e19-485f-43d0-9522-5fc1e2dd90cd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:04.538720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3e19-485f-43d0-9522-5fc1e2dd90cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:04.618337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b29d3e19-485f-43d0-9522-5fc1e2dd90cd None None] MFA Rules not processed for user `91bc1f3a0e9046bab7aa2c799ad46c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:04.619553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b29d3e19-485f-43d0-9522-5fc1e2dd90cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:04.743365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 695/1399] 10.222.0.39 () {58 vars in 1040 bytes} [Sun Sep 4 16:07:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:04.752406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dd70d-65fb-41b7-97ed-1a367a50d549 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:04.753977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e67dd70d-65fb-41b7-97ed-1a367a50d549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:04.896882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dd70d-65fb-41b7-97ed-1a367a50d549 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:04.998271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:05.018855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c4c8f-79c9-42bc-bfee-b67cac531adf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:05.019463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4c8f-79c9-42bc-bfee-b67cac531adf service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:05.019741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4c8f-79c9-42bc-bfee-b67cac531adf service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:05.020010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4c8f-79c9-42bc-bfee-b67cac531adf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:05.021665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2f0c4c8f-79c9-42bc-bfee-b67cac531adf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:05.104483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c4c8f-79c9-42bc-bfee-b67cac531adf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:05.106113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c4c8f-79c9-42bc-bfee-b67cac531adf service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:05.114794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 705/1400] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:07:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:07:05.560384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e43c5-ff86-41ce-9038-ffe05d9bd471 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:05.560941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e43c5-ff86-41ce-9038-ffe05d9bd471 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:05.560941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e43c5-ff86-41ce-9038-ffe05d9bd471 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:05.562573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 696/1401] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:05.568094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f5c3-c7cc-4882-8a6a-6d36dd22b5ca None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:05.568690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f5c3-c7cc-4882-8a6a-6d36dd22b5ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:05.568690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f5c3-c7cc-4882-8a6a-6d36dd22b5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:05.651034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4688f5c3-c7cc-4882-8a6a-6d36dd22b5ca None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:05.655082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4688f5c3-c7cc-4882-8a6a-6d36dd22b5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:05.729082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:05.765609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 706/1402] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:06.074719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e43c5-ff86-41ce-9038-ffe05d9bd471 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:06.076248 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1b8e43c5-ff86-41ce-9038-ffe05d9bd471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:06.143890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:06.243277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e43c5-ff86-41ce-9038-ffe05d9bd471 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:06.384538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc328744-07d4-4526-b210-5a49eefed6a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:06.385152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc328744-07d4-4526-b210-5a49eefed6a7 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:06.385787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc328744-07d4-4526-b210-5a49eefed6a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:06.386057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc328744-07d4-4526-b210-5a49eefed6a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:06.387677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bc328744-07d4-4526-b210-5a49eefed6a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:06.417469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:06.500147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc328744-07d4-4526-b210-5a49eefed6a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:06.501719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc328744-07d4-4526-b210-5a49eefed6a7 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:06.531983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 697/1403] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:09.144803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b9218-0df8-448a-9616-eaaade256a50 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:09.145778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b9218-0df8-448a-9616-eaaade256a50 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:09.146068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b9218-0df8-448a-9616-eaaade256a50 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:09.147763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 707/1404] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:09.153532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a8d30-f7e5-47f3-97d0-95a0058b5573 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:09.154356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a8d30-f7e5-47f3-97d0-95a0058b5573 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:09.154634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a8d30-f7e5-47f3-97d0-95a0058b5573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:09.225270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f52a8d30-f7e5-47f3-97d0-95a0058b5573 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:09.226432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f52a8d30-f7e5-47f3-97d0-95a0058b5573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:09.333843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:09.341539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 698/1405] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:09.639160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b9218-0df8-448a-9616-eaaade256a50 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:09.640677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-124b9218-0df8-448a-9616-eaaade256a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:09.818457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b9218-0df8-448a-9616-eaaade256a50 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:09.878935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:09.942649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34750ef7-3afa-4a5a-9710-56412ac84d68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:09.943256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34750ef7-3afa-4a5a-9710-56412ac84d68 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:09.943534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34750ef7-3afa-4a5a-9710-56412ac84d68 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:09.943801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34750ef7-3afa-4a5a-9710-56412ac84d68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:09.945397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-34750ef7-3afa-4a5a-9710-56412ac84d68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:10.045211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34750ef7-3afa-4a5a-9710-56412ac84d68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:10.049501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34750ef7-3afa-4a5a-9710-56412ac84d68 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:10.070110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 708/1406] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:12.244284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d6fea-6ab5-4e05-9c73-dc863b2f853e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:12.244798 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d6fea-6ab5-4e05-9c73-dc863b2f853e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:12.244798 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d6fea-6ab5-4e05-9c73-dc863b2f853e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:12.254049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 699/1407] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:07:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:12.255806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1e33-22b1-4309-8d34-f8c690b6dfde None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:12.256312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1e33-22b1-4309-8d34-f8c690b6dfde None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:12.256397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1e33-22b1-4309-8d34-f8c690b6dfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:12.353982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-93df1e33-22b1-4309-8d34-f8c690b6dfde None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:12.355256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-93df1e33-22b1-4309-8d34-f8c690b6dfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:12.370183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:12.474953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 709/1408] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:12.782799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d6fea-6ab5-4e05-9c73-dc863b2f853e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:12.784488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-827d6fea-6ab5-4e05-9c73-dc863b2f853e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:12.951222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d6fea-6ab5-4e05-9c73-dc863b2f853e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:13.025611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:13.079742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f90e7f-007c-464a-bf3b-16a0cc149262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:13.081028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f90e7f-007c-464a-bf3b-16a0cc149262 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:13.081501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f90e7f-007c-464a-bf3b-16a0cc149262 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:13.081806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f90e7f-007c-464a-bf3b-16a0cc149262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:13.083408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f5f90e7f-007c-464a-bf3b-16a0cc149262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:13.194107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f90e7f-007c-464a-bf3b-16a0cc149262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:13.195744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f90e7f-007c-464a-bf3b-16a0cc149262 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:13.215988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 700/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:22.583401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=38 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:23.213114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=64 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:28.488918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac69d-943a-4666-b4c9-47b42cdb8aa6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:28.489455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac69d-943a-4666-b4c9-47b42cdb8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:28.489455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac69d-943a-4666-b4c9-47b42cdb8aa6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:28.490799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 710/1410] 10.222.0.39 () {54 vars in 918 bytes} [Sun Sep 4 16:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:28.495944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075dd0c-a50c-42b4-b8de-a151884cd8e9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:28.496386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075dd0c-a50c-42b4-b8de-a151884cd8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:28.496489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075dd0c-a50c-42b4-b8de-a151884cd8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:28.579398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f075dd0c-a50c-42b4-b8de-a151884cd8e9 None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:28.580669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f075dd0c-a50c-42b4-b8de-a151884cd8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:28.691146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 701/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Sun Sep 4 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:28.705202 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857b53-ad0c-47e3-a8cc-c11dae442969 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:28.705478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857b53-ad0c-47e3-a8cc-c11dae442969 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:28.705775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857b53-ad0c-47e3-a8cc-c11dae442969 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:28.707104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 711/1412] 10.222.0.39 () {54 vars in 918 bytes} [Sun Sep 4 16:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:28.711606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcb104-8f06-4414-bdb7-1dcbd22205f7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:28.711889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcb104-8f06-4414-bdb7-1dcbd22205f7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:28.712158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcb104-8f06-4414-bdb7-1dcbd22205f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:28.785213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:28.787655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-50bcb104-8f06-4414-bdb7-1dcbd22205f7 None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:28.788862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-50bcb104-8f06-4414-bdb7-1dcbd22205f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:28.902753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 702/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Sun Sep 4 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:31.263424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718e01-8532-410d-9860-e9b8ca7bcbd6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:31.264099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718e01-8532-410d-9860-e9b8ca7bcbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:31.264099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718e01-8532-410d-9860-e9b8ca7bcbd6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:31.265318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:31.271272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06eec-879c-4ea9-9692-1486764098c2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:31.271704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06eec-879c-4ea9-9692-1486764098c2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:31.271807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06eec-879c-4ea9-9692-1486764098c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:31.338087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-38f06eec-879c-4ea9-9692-1486764098c2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:31.339322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-38f06eec-879c-4ea9-9692-1486764098c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:31.398878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:31.441482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:31.741619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718e01-8532-410d-9860-e9b8ca7bcbd6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:31.743138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dc718e01-8532-410d-9860-e9b8ca7bcbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:31.888758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718e01-8532-410d-9860-e9b8ca7bcbd6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:31.960093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:32.002232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d589a55-d62d-49d3-a3a7-7bbde1d1908d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:32.002785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d589a55-d62d-49d3-a3a7-7bbde1d1908d service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:32.003042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d589a55-d62d-49d3-a3a7-7bbde1d1908d service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:32.003313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d589a55-d62d-49d3-a3a7-7bbde1d1908d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:32.004903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d589a55-d62d-49d3-a3a7-7bbde1d1908d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:32.092022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d589a55-d62d-49d3-a3a7-7bbde1d1908d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:32.093629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d589a55-d62d-49d3-a3a7-7bbde1d1908d service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:32.111666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 713/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:33.029758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b271-5330-4868-abec-58b392e5a053 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:33.030355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b271-5330-4868-abec-58b392e5a053 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:33.030355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b271-5330-4868-abec-58b392e5a053 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:33.031636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:33.037593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274f7ea-beb7-4513-8b7e-1857d726b875 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:33.038016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274f7ea-beb7-4513-8b7e-1857d726b875 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:33.038128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274f7ea-beb7-4513-8b7e-1857d726b875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:33.113639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4274f7ea-beb7-4513-8b7e-1857d726b875 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:33.114837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4274f7ea-beb7-4513-8b7e-1857d726b875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:33.141232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:33.210106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:33.364871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b271-5330-4868-abec-58b392e5a053 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:33.366436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dc23b271-5330-4868-abec-58b392e5a053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:33.515799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b271-5330-4868-abec-58b392e5a053 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:33.556243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:33.628408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea1eaa-5348-4bed-b76e-da9a36ee214f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:33.628931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea1eaa-5348-4bed-b76e-da9a36ee214f service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:33.629209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea1eaa-5348-4bed-b76e-da9a36ee214f service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:33.629476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea1eaa-5348-4bed-b76e-da9a36ee214f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:33.631086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-19ea1eaa-5348-4bed-b76e-da9a36ee214f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:33.719657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea1eaa-5348-4bed-b76e-da9a36ee214f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:33.721279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea1eaa-5348-4bed-b76e-da9a36ee214f service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:33.738912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 705/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:34.631023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4ce08-6352-4681-ac75-ff4eca6e76de None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:34.631578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4ce08-6352-4681-ac75-ff4eca6e76de None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:34.631578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4ce08-6352-4681-ac75-ff4eca6e76de None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:34.633112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:34.638105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e99b-af5c-4009-a888-9b205eb3d482 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:34.638513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e99b-af5c-4009-a888-9b205eb3d482 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:34.638662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e99b-af5c-4009-a888-9b205eb3d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:34.692516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:34.713777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7aa4e99b-af5c-4009-a888-9b205eb3d482 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:34.714969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7aa4e99b-af5c-4009-a888-9b205eb3d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:34.810327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:34.960260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4ce08-6352-4681-ac75-ff4eca6e76de None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:34.961727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ef4ce08-6352-4681-ac75-ff4eca6e76de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:35.107954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4ce08-6352-4681-ac75-ff4eca6e76de None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:35.120199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:35.220089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6c6af-931c-4c1e-8f05-784462e4765a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:35.220764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6c6af-931c-4c1e-8f05-784462e4765a service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:35.221045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6c6af-931c-4c1e-8f05-784462e4765a service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:35.221475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6c6af-931c-4c1e-8f05-784462e4765a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:35.223141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5ee6c6af-931c-4c1e-8f05-784462e4765a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:35.311959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6c6af-931c-4c1e-8f05-784462e4765a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:35.313555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6c6af-931c-4c1e-8f05-784462e4765a service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:35.332990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 716/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:36.196883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92093c06-8e92-4304-836e-589bb41ce20e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:36.197466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92093c06-8e92-4304-836e-589bb41ce20e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:36.197466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92093c06-8e92-4304-836e-589bb41ce20e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:36.198822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:36.203633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1f7f0-0055-407c-b76a-34c2cd07c5f9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:36.204040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1f7f0-0055-407c-b76a-34c2cd07c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:36.204191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1f7f0-0055-407c-b76a-34c2cd07c5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:36.224672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:36.280555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e6a1f7f0-0055-407c-b76a-34c2cd07c5f9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:36.281776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e6a1f7f0-0055-407c-b76a-34c2cd07c5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:36.377779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:36.529284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92093c06-8e92-4304-836e-589bb41ce20e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:36.530735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-92093c06-8e92-4304-836e-589bb41ce20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:36.640215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:36.677538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92093c06-8e92-4304-836e-589bb41ce20e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:36.789067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bc4ff-4436-4e93-bc3f-66b545a8a3b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:36.789630 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bc4ff-4436-4e93-bc3f-66b545a8a3b0 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:36.789917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bc4ff-4436-4e93-bc3f-66b545a8a3b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:36.790193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bc4ff-4436-4e93-bc3f-66b545a8a3b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:36.791788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ce9bc4ff-4436-4e93-bc3f-66b545a8a3b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:36.866205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:36.879956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bc4ff-4436-4e93-bc3f-66b545a8a3b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:36.881550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bc4ff-4436-4e93-bc3f-66b545a8a3b0 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:36.902676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 708/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:37.763447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2bdc9-5f73-4dce-8c95-1bdcf33a5066 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:37.763982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2bdc9-5f73-4dce-8c95-1bdcf33a5066 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:37.763982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2bdc9-5f73-4dce-8c95-1bdcf33a5066 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:37.765270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:37.770699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93583570-7f14-4959-a608-d287885f3900 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:37.771149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93583570-7f14-4959-a608-d287885f3900 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:37.771258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93583570-7f14-4959-a608-d287885f3900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:37.846251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-93583570-7f14-4959-a608-d287885f3900 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:37.847459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-93583570-7f14-4959-a608-d287885f3900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:37.949632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:38.103332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2bdc9-5f73-4dce-8c95-1bdcf33a5066 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:38.104844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3bb2bdc9-5f73-4dce-8c95-1bdcf33a5066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:38.191165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:38.254860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2bdc9-5f73-4dce-8c95-1bdcf33a5066 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:38.366639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820aed6-3aa7-4f9e-b083-6c0970077917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:38.367177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820aed6-3aa7-4f9e-b083-6c0970077917 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:38.367452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820aed6-3aa7-4f9e-b083-6c0970077917 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:38.367718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820aed6-3aa7-4f9e-b083-6c0970077917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:38.369282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f820aed6-3aa7-4f9e-b083-6c0970077917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:38.417574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:38.457654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820aed6-3aa7-4f9e-b083-6c0970077917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:38.459246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820aed6-3aa7-4f9e-b083-6c0970077917 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:38.477810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 719/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:39.354611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd77f3-9c5b-4e14-b833-b428de08609f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:39.356184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd77f3-9c5b-4e14-b833-b428de08609f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:39.356184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd77f3-9c5b-4e14-b833-b428de08609f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:39.356546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:39.361647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb006-9d69-469d-8751-78892ac08645 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:39.362086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb006-9d69-469d-8751-78892ac08645 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:39.362238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb006-9d69-469d-8751-78892ac08645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:39.443561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-526eb006-9d69-469d-8751-78892ac08645 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:39.444810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-526eb006-9d69-469d-8751-78892ac08645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:39.541247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:39.691994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd77f3-9c5b-4e14-b833-b428de08609f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:39.693436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a0cd77f3-9c5b-4e14-b833-b428de08609f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:39.742933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:39.839203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd77f3-9c5b-4e14-b833-b428de08609f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:39.952185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed369f-288c-4f8b-9e97-2053f04f42b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:39.952708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed369f-288c-4f8b-9e97-2053f04f42b4 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:39.952985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed369f-288c-4f8b-9e97-2053f04f42b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:39.953255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed369f-288c-4f8b-9e97-2053f04f42b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:39.955539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cbed369f-288c-4f8b-9e97-2053f04f42b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:39.962344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:40.044834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed369f-288c-4f8b-9e97-2053f04f42b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:40.046452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed369f-288c-4f8b-9e97-2053f04f42b4 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:40.064559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 711/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:40.920124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6af4fb-7174-4cc8-af3f-ac327d8c3315 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:40.920667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6af4fb-7174-4cc8-af3f-ac327d8c3315 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:40.920667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6af4fb-7174-4cc8-af3f-ac327d8c3315 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:40.922070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:40.927549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca6c4-74a9-4182-b2e6-487699392ad1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:40.927986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca6c4-74a9-4182-b2e6-487699392ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:40.928138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca6c4-74a9-4182-b2e6-487699392ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:41.003145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f02ca6c4-74a9-4182-b2e6-487699392ad1 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:41.004334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f02ca6c4-74a9-4182-b2e6-487699392ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:41.072021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:41.100875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:41.253889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6af4fb-7174-4cc8-af3f-ac327d8c3315 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:41.255336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bf6af4fb-7174-4cc8-af3f-ac327d8c3315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:41.273498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:41.402834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6af4fb-7174-4cc8-af3f-ac327d8c3315 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:41.489679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:41.515549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053f4ff-adfb-47a0-8bf1-dbda847ca533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:41.516192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053f4ff-adfb-47a0-8bf1-dbda847ca533 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:41.516476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053f4ff-adfb-47a0-8bf1-dbda847ca533 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:41.517409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053f4ff-adfb-47a0-8bf1-dbda847ca533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:41.519190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1053f4ff-adfb-47a0-8bf1-dbda847ca533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:41.607296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053f4ff-adfb-47a0-8bf1-dbda847ca533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:41.608880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053f4ff-adfb-47a0-8bf1-dbda847ca533 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:41.627568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 722/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:42.499840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f54c9-947f-44b8-8f59-768db230345c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:42.500385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f54c9-947f-44b8-8f59-768db230345c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:42.500385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f54c9-947f-44b8-8f59-768db230345c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:42.502348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:42.507236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37946cf2-0e34-4378-a5fd-7321e18f5f7d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:42.507934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37946cf2-0e34-4378-a5fd-7321e18f5f7d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:42.508074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37946cf2-0e34-4378-a5fd-7321e18f5f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:42.585380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-37946cf2-0e34-4378-a5fd-7321e18f5f7d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:42.586623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-37946cf2-0e34-4378-a5fd-7321e18f5f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:42.628467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:42.682160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:42.834469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f54c9-947f-44b8-8f59-768db230345c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:42.835991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f61f54c9-947f-44b8-8f59-768db230345c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:42.980886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f54c9-947f-44b8-8f59-768db230345c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:43.036334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:43.095296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4956617-9e75-40ec-94e3-b6dc7bfa58f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:43.095835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956617-9e75-40ec-94e3-b6dc7bfa58f7 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:43.096117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956617-9e75-40ec-94e3-b6dc7bfa58f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:43.096388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956617-9e75-40ec-94e3-b6dc7bfa58f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:43.098023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a4956617-9e75-40ec-94e3-b6dc7bfa58f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:43.186950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4956617-9e75-40ec-94e3-b6dc7bfa58f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:43.189113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4956617-9e75-40ec-94e3-b6dc7bfa58f7 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:43.207747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 714/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:44.078838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7748890-a4ba-4892-80ee-6c937018a0be None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:44.079907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7748890-a4ba-4892-80ee-6c937018a0be None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:44.080029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7748890-a4ba-4892-80ee-6c937018a0be None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:44.081357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:44.086247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b26ea-8350-4078-88d4-239a86d2343b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:44.086629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b26ea-8350-4078-88d4-239a86d2343b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:44.086802 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b26ea-8350-4078-88d4-239a86d2343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:44.163488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-407b26ea-8350-4078-88d4-239a86d2343b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:44.164706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-407b26ea-8350-4078-88d4-239a86d2343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:44.166798 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:44.266086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:44.415934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7748890-a4ba-4892-80ee-6c937018a0be None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:44.417453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c7748890-a4ba-4892-80ee-6c937018a0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:44.566874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7748890-a4ba-4892-80ee-6c937018a0be None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:44.593809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:44.681804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842e9f6-ba0d-4aca-b8c7-47b408e730c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:44.682417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e9f6-ba0d-4aca-b8c7-47b408e730c6 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:44.682696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e9f6-ba0d-4aca-b8c7-47b408e730c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:44.682971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e9f6-ba0d-4aca-b8c7-47b408e730c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:44.684568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6842e9f6-ba0d-4aca-b8c7-47b408e730c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:44.778663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842e9f6-ba0d-4aca-b8c7-47b408e730c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:44.780329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842e9f6-ba0d-4aca-b8c7-47b408e730c6 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:44.800791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 725/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:46.626807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ecd49-5e1e-448b-937f-d63add7d17f3 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:46.627372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ecd49-5e1e-448b-937f-d63add7d17f3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:46.627372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ecd49-5e1e-448b-937f-d63add7d17f3 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:46.628705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:46.634137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f243a-5361-40c8-9120-14a4d1f9e7cf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:46.634524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f243a-5361-40c8-9120-14a4d1f9e7cf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:46.634682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f243a-5361-40c8-9120-14a4d1f9e7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:46.675538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:46.700438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-343f243a-5361-40c8-9120-14a4d1f9e7cf None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:46.701693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-343f243a-5361-40c8-9120-14a4d1f9e7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:46.797465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:47.086228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ecd49-5e1e-448b-937f-d63add7d17f3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:47.087861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-756ecd49-5e1e-448b-937f-d63add7d17f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:47.190727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ecd49-5e1e-448b-937f-d63add7d17f3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:47.222810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:47.264086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1f610-2eaf-417e-8e24-60e65b1f74e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:47.264690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f610-2eaf-417e-8e24-60e65b1f74e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:47.264971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f610-2eaf-417e-8e24-60e65b1f74e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:47.265245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f610-2eaf-417e-8e24-60e65b1f74e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:47.267582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dfe1f610-2eaf-417e-8e24-60e65b1f74e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:47.379466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1f610-2eaf-417e-8e24-60e65b1f74e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:47.380930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1f610-2eaf-417e-8e24-60e65b1f74e3 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:47.400009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 717/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:49.470564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a0258-4445-4de1-9157-2c5cb0a93153 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:49.471175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a0258-4445-4de1-9157-2c5cb0a93153 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:49.471175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a0258-4445-4de1-9157-2c5cb0a93153 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:49.472514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:49.477795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d6d31-51a7-49b4-baee-a06b57e558c8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:49.478206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d6d31-51a7-49b4-baee-a06b57e558c8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:49.478357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d6d31-51a7-49b4-baee-a06b57e558c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:49.545695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-074d6d31-51a7-49b4-baee-a06b57e558c8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:49.546929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-074d6d31-51a7-49b4-baee-a06b57e558c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:49.560791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:49.643638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:07:49.933608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a0258-4445-4de1-9157-2c5cb0a93153 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:49.935138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bd8a0258-4445-4de1-9157-2c5cb0a93153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:50.036595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a0258-4445-4de1-9157-2c5cb0a93153 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:50.048922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:50.109623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f976d-ee80-4bed-8031-37d09a5ea633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:50.110770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f976d-ee80-4bed-8031-37d09a5ea633 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:50.111168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f976d-ee80-4bed-8031-37d09a5ea633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:50.111411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f976d-ee80-4bed-8031-37d09a5ea633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:50.113043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4a1f976d-ee80-4bed-8031-37d09a5ea633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:50.221853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f976d-ee80-4bed-8031-37d09a5ea633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:50.223063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f976d-ee80-4bed-8031-37d09a5ea633 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:50.241184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 728/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:52.519485 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d3c1b-1780-49e6-9298-d5dfc377e111 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:52.520022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d3c1b-1780-49e6-9298-d5dfc377e111 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:52.520022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d3c1b-1780-49e6-9298-d5dfc377e111 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:52.521308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:52.527769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922e016-2303-473f-9877-b86937b4a995 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:52.528316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922e016-2303-473f-9877-b86937b4a995 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:52.528423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922e016-2303-473f-9877-b86937b4a995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:52.589358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:52.607200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b922e016-2303-473f-9877-b86937b4a995 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:52.608407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b922e016-2303-473f-9877-b86937b4a995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:52.703173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:52.987466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d3c1b-1780-49e6-9298-d5dfc377e111 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:52.989040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fa3d3c1b-1780-49e6-9298-d5dfc377e111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:53.092207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d3c1b-1780-49e6-9298-d5dfc377e111 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:53.131573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:53.169347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c3f4c-1078-4a3f-81d2-7c0ec530d3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:53.170221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c3f4c-1078-4a3f-81d2-7c0ec530d3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:53.170683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c3f4c-1078-4a3f-81d2-7c0ec530d3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:53.171236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c3f4c-1078-4a3f-81d2-7c0ec530d3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:53.172870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-684c3f4c-1078-4a3f-81d2-7c0ec530d3b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:53.279706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c3f4c-1078-4a3f-81d2-7c0ec530d3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:53.281069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c3f4c-1078-4a3f-81d2-7c0ec530d3b8 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:53.299655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 720/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:55.335449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0628724-109b-4988-b6ed-5adf93b6b97a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:55.336008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0628724-109b-4988-b6ed-5adf93b6b97a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:55.336008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0628724-109b-4988-b6ed-5adf93b6b97a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:55.337338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:55.342820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bf29-bfa3-44af-9c8c-38e460f10ab4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:55.343239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bf29-bfa3-44af-9c8c-38e460f10ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:55.343431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bf29-bfa3-44af-9c8c-38e460f10ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:55.409399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3dd2bf29-bfa3-44af-9c8c-38e460f10ab4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:55.410599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3dd2bf29-bfa3-44af-9c8c-38e460f10ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:55.430972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:55.507850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:55.791770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0628724-109b-4988-b6ed-5adf93b6b97a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:55.793310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b0628724-109b-4988-b6ed-5adf93b6b97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:55.896285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0628724-109b-4988-b6ed-5adf93b6b97a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:55.916441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:55.968923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad6911-fbaa-4b6c-afda-d7b515301735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:55.970277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6911-fbaa-4b6c-afda-d7b515301735 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:55.970570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6911-fbaa-4b6c-afda-d7b515301735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:55.970917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6911-fbaa-4b6c-afda-d7b515301735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:55.972502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b0ad6911-fbaa-4b6c-afda-d7b515301735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:56.081771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad6911-fbaa-4b6c-afda-d7b515301735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:56.083150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad6911-fbaa-4b6c-afda-d7b515301735 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:56.101347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 731/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:07:58.067506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc1b6-a92b-41f6-8b0d-c9b40be22156 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:58.068082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc1b6-a92b-41f6-8b0d-c9b40be22156 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:58.068183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc1b6-a92b-41f6-8b0d-c9b40be22156 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:58.069510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:07:58.074905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581a43-9866-43bc-a1f3-97e908f7084e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:58.075336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581a43-9866-43bc-a1f3-97e908f7084e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:58.075453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581a43-9866-43bc-a1f3-97e908f7084e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:58.139944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:58.143580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-18581a43-9866-43bc-a1f3-97e908f7084e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:07:58.144795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-18581a43-9866-43bc-a1f3-97e908f7084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:58.241744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:07:58.533206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dc1b6-a92b-41f6-8b0d-c9b40be22156 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:07:58.534844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-834dc1b6-a92b-41f6-8b0d-c9b40be22156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:58.642565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dc1b6-a92b-41f6-8b0d-c9b40be22156 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:07:58.691547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:07:58.716267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268f119-4556-42cb-bb0e-594737c68c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:07:58.716923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268f119-4556-42cb-bb0e-594737c68c8f service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:07:58.717201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268f119-4556-42cb-bb0e-594737c68c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:07:58.717472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268f119-4556-42cb-bb0e-594737c68c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:07:58.719098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e268f119-4556-42cb-bb0e-594737c68c8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:07:58.827005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268f119-4556-42cb-bb0e-594737c68c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:07:58.828440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268f119-4556-42cb-bb0e-594737c68c8f service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:07:58.846950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 723/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:00.839629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9bf70-60a6-4ef4-b325-25cf230c8450 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:00.840204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9bf70-60a6-4ef4-b325-25cf230c8450 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:00.840204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9bf70-60a6-4ef4-b325-25cf230c8450 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:00.841534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:00.847398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543252a-00b3-4494-82b7-b6f21ae96f8c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:00.847801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543252a-00b3-4494-82b7-b6f21ae96f8c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:00.847953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543252a-00b3-4494-82b7-b6f21ae96f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:00.921462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b543252a-00b3-4494-82b7-b6f21ae96f8c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:00.923326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b543252a-00b3-4494-82b7-b6f21ae96f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:00.951814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:01.018428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:01.329780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9bf70-60a6-4ef4-b325-25cf230c8450 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:01.331851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ed9bf70-60a6-4ef4-b325-25cf230c8450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:01.470610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9bf70-60a6-4ef4-b325-25cf230c8450 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:01.511685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:01.576617 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac7688-f031-4c22-bf02-6665240032bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:01.577231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7688-f031-4c22-bf02-6665240032bd service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:01.577508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7688-f031-4c22-bf02-6665240032bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:01.577811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7688-f031-4c22-bf02-6665240032bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:01.580896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-32ac7688-f031-4c22-bf02-6665240032bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:01.710455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac7688-f031-4c22-bf02-6665240032bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:01.712784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac7688-f031-4c22-bf02-6665240032bd service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:01.737767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 734/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:03.761523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eba135-a6cd-4ced-8ccb-2ea62dcf9f76 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:03.762093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eba135-a6cd-4ced-8ccb-2ea62dcf9f76 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:03.762093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eba135-a6cd-4ced-8ccb-2ea62dcf9f76 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:03.763383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:03.768533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9d5af-2761-467f-8742-db7a4ed3b25f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:03.768959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9d5af-2761-467f-8742-db7a4ed3b25f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:03.769096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9d5af-2761-467f-8742-db7a4ed3b25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:03.834966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f9b9d5af-2761-467f-8742-db7a4ed3b25f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:03.836178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f9b9d5af-2761-467f-8742-db7a4ed3b25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:03.845152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:03.934396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:04.216281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eba135-a6cd-4ced-8ccb-2ea62dcf9f76 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:04.217844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-00eba135-a6cd-4ced-8ccb-2ea62dcf9f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:04.322278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eba135-a6cd-4ced-8ccb-2ea62dcf9f76 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:04.378758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:04.397104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934fbd-87cd-4f0d-be6e-53258d5ab8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:04.397754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934fbd-87cd-4f0d-be6e-53258d5ab8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:04.399027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934fbd-87cd-4f0d-be6e-53258d5ab8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:04.399027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934fbd-87cd-4f0d-be6e-53258d5ab8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:04.399892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-46934fbd-87cd-4f0d-be6e-53258d5ab8e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:04.509290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46934fbd-87cd-4f0d-be6e-53258d5ab8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:04.510678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46934fbd-87cd-4f0d-be6e-53258d5ab8e2 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:04.529354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 726/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:06.490425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0725-49e2-498e-858f-f0af711b6cf6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:06.490954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0725-49e2-498e-858f-f0af711b6cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:06.490954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0725-49e2-498e-858f-f0af711b6cf6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:06.492268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:08:06.497388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ffb2c-078c-4f6d-a3d6-a55833c065c3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:06.497885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ffb2c-078c-4f6d-a3d6-a55833c065c3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:06.497994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ffb2c-078c-4f6d-a3d6-a55833c065c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:06.568604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-739ffb2c-078c-4f6d-a3d6-a55833c065c3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:06.569855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-739ffb2c-078c-4f6d-a3d6-a55833c065c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:06.608879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:06.671429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:06.953321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0725-49e2-498e-858f-f0af711b6cf6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:06.955457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-99ed0725-49e2-498e-858f-f0af711b6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:07.058140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0725-49e2-498e-858f-f0af711b6cf6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:07.094960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:07.130898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e925c2-6a3f-4e27-8576-6fd6e42b7db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:07.131505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e925c2-6a3f-4e27-8576-6fd6e42b7db4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:07.131786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e925c2-6a3f-4e27-8576-6fd6e42b7db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:07.132059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e925c2-6a3f-4e27-8576-6fd6e42b7db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:07.133939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-92e925c2-6a3f-4e27-8576-6fd6e42b7db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:07.241099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e925c2-6a3f-4e27-8576-6fd6e42b7db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:07.242501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e925c2-6a3f-4e27-8576-6fd6e42b7db4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:07.261313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 737/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:09.171102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c068a-7707-4159-8708-b9d37fe34f3c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:09.171650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c068a-7707-4159-8708-b9d37fe34f3c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:09.171650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c068a-7707-4159-8708-b9d37fe34f3c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:09.173943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:09.179236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5a13c-c359-445f-b3e8-58a4496c6c76 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:09.179661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5a13c-c359-445f-b3e8-58a4496c6c76 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:09.179815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5a13c-c359-445f-b3e8-58a4496c6c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:09.246554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-16e5a13c-c359-445f-b3e8-58a4496c6c76 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:09.247745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16e5a13c-c359-445f-b3e8-58a4496c6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:09.267599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:09.350456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:09.630643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c068a-7707-4159-8708-b9d37fe34f3c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:09.632179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4a3c068a-7707-4159-8708-b9d37fe34f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:09.735313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c068a-7707-4159-8708-b9d37fe34f3c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:09.802582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:09.809150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39620c06-d407-4bb3-9a49-752e3d694a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:09.809790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620c06-d407-4bb3-9a49-752e3d694a24 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:09.810070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620c06-d407-4bb3-9a49-752e3d694a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:09.810338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39620c06-d407-4bb3-9a49-752e3d694a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:09.811928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-39620c06-d407-4bb3-9a49-752e3d694a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:09.918580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39620c06-d407-4bb3-9a49-752e3d694a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:09.919923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39620c06-d407-4bb3-9a49-752e3d694a24 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:09.937718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 729/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:11.913530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d397695-c5d4-4e40-a2e3-291257dde631 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:11.914114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d397695-c5d4-4e40-a2e3-291257dde631 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:11.914114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d397695-c5d4-4e40-a2e3-291257dde631 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:11.915491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:11.920624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bd1b3-1b6f-4845-a015-ce53bae7e8fc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:11.921046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bd1b3-1b6f-4845-a015-ce53bae7e8fc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:11.921204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bd1b3-1b6f-4845-a015-ce53bae7e8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:11.987294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-018bd1b3-1b6f-4845-a015-ce53bae7e8fc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:11.988500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-018bd1b3-1b6f-4845-a015-ce53bae7e8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:12.036283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:12.083669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:12.367591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d397695-c5d4-4e40-a2e3-291257dde631 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:12.369117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9d397695-c5d4-4e40-a2e3-291257dde631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:12.475318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d397695-c5d4-4e40-a2e3-291257dde631 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:12.522592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:12.548545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535805b9-1650-48a0-82ae-0711f8065dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:12.549152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535805b9-1650-48a0-82ae-0711f8065dda service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:12.549436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535805b9-1650-48a0-82ae-0711f8065dda service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:12.549739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535805b9-1650-48a0-82ae-0711f8065dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:12.551326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-535805b9-1650-48a0-82ae-0711f8065dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:12.660152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535805b9-1650-48a0-82ae-0711f8065dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:12.661316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535805b9-1650-48a0-82ae-0711f8065dda service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:12.679427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 740/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:22.089791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=22 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:22.677150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=66 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:32.270218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b8de3-fc19-46a0-907b-9cf22db36fb9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:32.270700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b8de3-fc19-46a0-907b-9cf22db36fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:32.270809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b8de3-fc19-46a0-907b-9cf22db36fb9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:32.272110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 731/1471] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:32.277316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e9f1e-1448-4fda-a3ec-722588056fd9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:32.277779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e9f1e-1448-4fda-a3ec-722588056fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:32.277893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e9f1e-1448-4fda-a3ec-722588056fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:32.343064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0b8e9f1e-1448-4fda-a3ec-722588056fd9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:32.344884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b8e9f1e-1448-4fda-a3ec-722588056fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:32.440096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 741/1472] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:32.752449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b51ad-391f-4105-a757-b2205bc4690f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:32.752735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b51ad-391f-4105-a757-b2205bc4690f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:32.753537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b51ad-391f-4105-a757-b2205bc4690f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:32.754924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 732/1473] 10.222.0.39 () {54 vars in 961 bytes} [Sun Sep 4 16:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:32.760053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3089b4-aa78-496f-93ce-f937dc761f06 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:32.760342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3089b4-aa78-496f-93ce-f937dc761f06 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:32.760647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3089b4-aa78-496f-93ce-f937dc761f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:32.835984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ed3089b4-aa78-496f-93ce-f937dc761f06 None None] MFA Rules not processed for user `c95204bd338a4a99a510a67c4ab97532`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:32.837199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed3089b4-aa78-496f-93ce-f937dc761f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:32.857920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:32.914940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 742/1474] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Sep 4 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:32.922992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b51ad-391f-4105-a757-b2205bc4690f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:32.924559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-246b51ad-391f-4105-a757-b2205bc4690f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:33.031600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b51ad-391f-4105-a757-b2205bc4690f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:33.105502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c4e24-775f-4c74-b9ef-16fe62c2f52f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95204bd338a4a99a510a67c4ab97532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:33.106136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c4e24-775f-4c74-b9ef-16fe62c2f52f service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:33.106416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c4e24-775f-4c74-b9ef-16fe62c2f52f service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:33.106689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c4e24-775f-4c74-b9ef-16fe62c2f52f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:33.108309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-745c4e24-775f-4c74-b9ef-16fe62c2f52f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:33.163731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:33.221302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c4e24-775f-4c74-b9ef-16fe62c2f52f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:33.222764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c4e24-775f-4c74-b9ef-16fe62c2f52f service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:33.240830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 733/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:35.230402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15a88-9743-4194-a554-0fdbb1f04182 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:35.230955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15a88-9743-4194-a554-0fdbb1f04182 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:35.230955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15a88-9743-4194-a554-0fdbb1f04182 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:35.232370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 743/1476] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:35.238312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a5b6-6519-4de2-9177-e79f23e5c55d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:35.238749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a5b6-6519-4de2-9177-e79f23e5c55d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:35.238897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a5b6-6519-4de2-9177-e79f23e5c55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:35.306425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4058a5b6-6519-4de2-9177-e79f23e5c55d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:35.307646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4058a5b6-6519-4de2-9177-e79f23e5c55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:35.407461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 734/1477] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:35.694330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15a88-9743-4194-a554-0fdbb1f04182 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:35.695858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f8b15a88-9743-4194-a554-0fdbb1f04182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:35.820205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15a88-9743-4194-a554-0fdbb1f04182 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:35.868675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:35.912772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ec438-6d4e-4a10-ba5c-596e9553786a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:35.913376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ec438-6d4e-4a10-ba5c-596e9553786a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:35.913686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ec438-6d4e-4a10-ba5c-596e9553786a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:35.913958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ec438-6d4e-4a10-ba5c-596e9553786a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:35.914591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ec438-6d4e-4a10-ba5c-596e9553786a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:35.916364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-d47ec438-6d4e-4a10-ba5c-596e9553786a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:08:35.916364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:08:35.916364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:08:35.917510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ec438-6d4e-4a10-ba5c-596e9553786a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:35.936322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 744/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Sep 4 16:08:35 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:08:37.923408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0f86-cc8a-4a43-9c68-2c5c5a915f28 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:37.923973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0f86-cc8a-4a43-9c68-2c5c5a915f28 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:37.923973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0f86-cc8a-4a43-9c68-2c5c5a915f28 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:37.925281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 735/1479] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:37.930709 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691626b-a9a0-402f-af2d-8858987c7a16 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:37.931139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691626b-a9a0-402f-af2d-8858987c7a16 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:37.931260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691626b-a9a0-402f-af2d-8858987c7a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:38.119034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2691626b-a9a0-402f-af2d-8858987c7a16 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:38.120276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2691626b-a9a0-402f-af2d-8858987c7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:38.217136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 745/1480] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:38.510227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0f86-cc8a-4a43-9c68-2c5c5a915f28 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:38.511736 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-680b0f86-cc8a-4a43-9c68-2c5c5a915f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:38.522489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:38.663078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0f86-cc8a-4a43-9c68-2c5c5a915f28 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:38.742562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:38.778999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de33cc-6e93-4477-a24c-2d448a3b838e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:38.779527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de33cc-6e93-4477-a24c-2d448a3b838e service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:38.779805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de33cc-6e93-4477-a24c-2d448a3b838e service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:38.780076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de33cc-6e93-4477-a24c-2d448a3b838e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:38.781910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-68de33cc-6e93-4477-a24c-2d448a3b838e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:38.870271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de33cc-6e93-4477-a24c-2d448a3b838e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:38.871841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de33cc-6e93-4477-a24c-2d448a3b838e service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:38.892796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 736/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:41.055326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef431-f2ce-4da2-b4d2-74d99289ecdb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:41.055898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef431-f2ce-4da2-b4d2-74d99289ecdb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:41.055898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef431-f2ce-4da2-b4d2-74d99289ecdb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:41.057276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 746/1482] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:41.063136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267d51b-3464-4a9d-ac6e-35e2e179a51f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:41.063521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267d51b-3464-4a9d-ac6e-35e2e179a51f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:41.063696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267d51b-3464-4a9d-ac6e-35e2e179a51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:41.130462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b267d51b-3464-4a9d-ac6e-35e2e179a51f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:41.131668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b267d51b-3464-4a9d-ac6e-35e2e179a51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:41.179505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:41.230330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 737/1483] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:41.519983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fef431-f2ce-4da2-b4d2-74d99289ecdb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:41.521618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75fef431-f2ce-4da2-b4d2-74d99289ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:41.545025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:41.624250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fef431-f2ce-4da2-b4d2-74d99289ecdb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:41.698605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f784df8-960e-4eb7-af42-c3ae85a4e20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:41.699214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f784df8-960e-4eb7-af42-c3ae85a4e20d service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:41.699494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f784df8-960e-4eb7-af42-c3ae85a4e20d service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:41.699807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f784df8-960e-4eb7-af42-c3ae85a4e20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:41.701407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9f784df8-960e-4eb7-af42-c3ae85a4e20d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:41.775259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:41.810693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f784df8-960e-4eb7-af42-c3ae85a4e20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:41.812380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f784df8-960e-4eb7-af42-c3ae85a4e20d service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:41.831442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 747/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:43.779093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d509050-a740-406e-b089-a0cd5f21a218 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:43.779664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d509050-a740-406e-b089-a0cd5f21a218 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:43.779664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d509050-a740-406e-b089-a0cd5f21a218 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:43.781029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 738/1485] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:43.786326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087c99-ab6a-4169-b1f4-b191b605e910 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:43.786729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087c99-ab6a-4169-b1f4-b191b605e910 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:43.786877 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087c99-ab6a-4169-b1f4-b191b605e910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:43.851920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-93087c99-ab6a-4169-b1f4-b191b605e910 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:43.853106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-93087c99-ab6a-4169-b1f4-b191b605e910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:43.948182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 748/1486] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:08:44.232937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d509050-a740-406e-b089-a0cd5f21a218 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:44.235106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8d509050-a740-406e-b089-a0cd5f21a218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:44.337441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d509050-a740-406e-b089-a0cd5f21a218 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:44.409135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:44.410948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3625a8-38b4-431c-b558-503069eb2524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:44.411547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3625a8-38b4-431c-b558-503069eb2524 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:44.411828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3625a8-38b4-431c-b558-503069eb2524 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:44.412100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3625a8-38b4-431c-b558-503069eb2524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:44.413724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0e3625a8-38b4-431c-b558-503069eb2524 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:44.521164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3625a8-38b4-431c-b558-503069eb2524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:44.523336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3625a8-38b4-431c-b558-503069eb2524 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:44.542474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 739/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:46.675166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721d22-d88e-4b15-85dd-20619af65704 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:46.675735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721d22-d88e-4b15-85dd-20619af65704 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:46.675735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721d22-d88e-4b15-85dd-20619af65704 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:46.677048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 749/1488] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:46.682344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d52857-89c5-4da1-80e1-1b6f74b4a1e6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:46.682735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d52857-89c5-4da1-80e1-1b6f74b4a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:46.682908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d52857-89c5-4da1-80e1-1b6f74b4a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:46.750616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f8d52857-89c5-4da1-80e1-1b6f74b4a1e6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:46.751829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f8d52857-89c5-4da1-80e1-1b6f74b4a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:46.808895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:46.857087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 740/1489] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:08:47.151486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721d22-d88e-4b15-85dd-20619af65704 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:47.153019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-54721d22-d88e-4b15-85dd-20619af65704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:47.190256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:47.256949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721d22-d88e-4b15-85dd-20619af65704 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:47.328398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b1223-83df-418f-8475-09bd701ad198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:47.329096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1223-83df-418f-8475-09bd701ad198 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:47.329379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1223-83df-418f-8475-09bd701ad198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:47.329679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1223-83df-418f-8475-09bd701ad198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:47.331270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-353b1223-83df-418f-8475-09bd701ad198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:47.410899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:47.439676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b1223-83df-418f-8475-09bd701ad198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:47.441111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b1223-83df-418f-8475-09bd701ad198 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:47.458682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 750/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:49.754976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e17ff5-f26f-4f23-b1e0-5e08f96f908c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:49.755637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e17ff5-f26f-4f23-b1e0-5e08f96f908c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:49.755637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e17ff5-f26f-4f23-b1e0-5e08f96f908c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:49.756881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 741/1491] 10.222.0.39 () {54 vars in 927 bytes} [Sun Sep 4 16:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:08:49.762715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ff5b7-72fc-4b85-8006-0c63d88c6b66 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:49.763146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ff5b7-72fc-4b85-8006-0c63d88c6b66 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:49.763280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ff5b7-72fc-4b85-8006-0c63d88c6b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:49.828773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5a7ff5b7-72fc-4b85-8006-0c63d88c6b66 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:08:49.830009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5a7ff5b7-72fc-4b85-8006-0c63d88c6b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:49.927512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 751/1492] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Sep 4 16:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:08:50.215232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e17ff5-f26f-4f23-b1e0-5e08f96f908c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:08:50.216771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e3e17ff5-f26f-4f23-b1e0-5e08f96f908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:50.321361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e17ff5-f26f-4f23-b1e0-5e08f96f908c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:08:50.396386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4272db-9813-4a6e-b249-996335f2d844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:08:50.397003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4272db-9813-4a6e-b249-996335f2d844 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:08:50.397283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4272db-9813-4a6e-b249-996335f2d844 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:08:50.397580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4272db-9813-4a6e-b249-996335f2d844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:08:50.399193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9a4272db-9813-4a6e-b249-996335f2d844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:08:50.408488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:08:50.509126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4272db-9813-4a6e-b249-996335f2d844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:08:50.510515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4272db-9813-4a6e-b249-996335f2d844 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:08:50.529086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 742/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:08:59.925730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=82 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:09:00.526916 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=50 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:12.209419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:12.210083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:12.210083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:12.283741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:10:12.284937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:12.382170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 752/1494] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:10:12.390304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f616-9755-40ea-938f-9c8271910e91 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:12.390777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f616-9755-40ea-938f-9c8271910e91 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:12.390881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f616-9755-40ea-938f-9c8271910e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:12.467824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d210f616-9755-40ea-938f-9c8271910e91 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:10:12.469155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d210f616-9755-40ea-938f-9c8271910e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:12.567699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 743/1495] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:10:12.572348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:12.573594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:12.584802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bcd0f-3918-471a-a029-1720331e0835 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:12.645331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:12.678601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:12.679202 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:12.679478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:12.679747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:12.680385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:10:12.680714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:12.681761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:12.681761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:10:12.681761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:10:12.682650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:12.693801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 753/1496] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:10:12.701971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210f616-9755-40ea-938f-9c8271910e91 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:12.711473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210f616-9755-40ea-938f-9c8271910e91 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:12.771837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:12.805045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:12.805667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:12.805947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:12.806239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:12.807088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:12.809374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:12.809374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Sep 04 16:10:12.809374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: Sep 04 16:10:12.810654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:12.951639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 744/1497] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:10:12 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:10:12.956052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:12.957648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:12.992483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71bdf8-8073-4ef4-a4e7-b7d23c4f1ac1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:13.088074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:13.088691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:13.088973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:13.089247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:13.089916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:13.090938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:13.090938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:10:13.090938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:10:13.092149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:13.233492 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 754/1498] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:10:12 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:10:13.237657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:13.239150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:13.304492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ddcb9-e30c-402e-a0af-38190372c690 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:13.388743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:13.399483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:13.410720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:13.410720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:13.410720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:13.410720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:13.410720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:13.410720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Sep 04 16:10:13.410720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: Sep 04 16:10:13.411343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:13.418747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 745/1499] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:10:13.423248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:13.424780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:13.436138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b4cfe-ebac-4b5f-992c-b86f5f2306ea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:13.461926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:13.530964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cce1c-89cf-4445-af26-c8e17cd260dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:13.531738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cce1c-89cf-4445-af26-c8e17cd260dd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:13.532010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cce1c-89cf-4445-af26-c8e17cd260dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:13.532301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cce1c-89cf-4445-af26-c8e17cd260dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:13.532939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cce1c-89cf-4445-af26-c8e17cd260dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:13.534038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-b78cce1c-89cf-4445-af26-c8e17cd260dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:13.534038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:10:13.534038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:10:13.535015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cce1c-89cf-4445-af26-c8e17cd260dd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:13.547731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 755/1500] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:10:13.551948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:13.553493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:13.587683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385430f-16c3-4292-81b7-872dbc8cd252 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:13.681635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f7718-1aa9-4666-a61c-37b57a598c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:13.682216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f7718-1aa9-4666-a61c-37b57a598c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:13.682502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f7718-1aa9-4666-a61c-37b57a598c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:13.682782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f7718-1aa9-4666-a61c-37b57a598c66 admin admin] PATH_INFO: `/v3/projects/7323203d53dd486ba94de7cc575332ff/users/3fbc34d0ca1e4a3d9608e01779b4e062/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:13.707054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f7718-1aa9-4666-a61c-37b57a598c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7323203d53dd486ba94de7cc575332ff, user_id=3fbc34d0ca1e4a3d9608e01779b4e062, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:13.708079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-bf3f7718-1aa9-4666-a61c-37b57a598c66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:13.708079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Sep 04 16:10:13.708079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: Sep 04 16:10:13.710057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f7718-1aa9-4666-a61c-37b57a598c66 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:13.784781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 746/1501] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:10:13 2022] PUT /identity/v3/projects/7323203d53dd486ba94de7cc575332ff/users/3fbc34d0ca1e4a3d9608e01779b4e062/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:10:13.797751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7e8e-e6a1-4a7d-b643-6104ffd77673 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:13.798035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7e8e-e6a1-4a7d-b643-6104ffd77673 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:13.798338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7e8e-e6a1-4a7d-b643-6104ffd77673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:13.854914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2e2d7e8e-e6a1-4a7d-b643-6104ffd77673 None None] MFA Rules not processed for user `3fbc34d0ca1e4a3d9608e01779b4e062`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:10:13.856133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2e2d7e8e-e6a1-4a7d-b643-6104ffd77673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:13.933771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 756/1502] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:10:13.947624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5d9c-a2fc-44ee-a23c-5e0abade637d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:13.947908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5d9c-a2fc-44ee-a23c-5e0abade637d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:13.948207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5d9c-a2fc-44ee-a23c-5e0abade637d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:13.949550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 747/1503] 10.222.0.39 () {54 vars in 964 bytes} [Sun Sep 4 16:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:10:13.954569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24adcf8-5923-4226-8e4e-0d7447fae907 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:13.954862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24adcf8-5923-4226-8e4e-0d7447fae907 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:13.955143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24adcf8-5923-4226-8e4e-0d7447fae907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:13.986570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:14.035191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b24adcf8-5923-4226-8e4e-0d7447fae907 None None] MFA Rules not processed for user `c029d59023b04bf8b8bf9d4d5eee7390`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:10:14.036443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b24adcf8-5923-4226-8e4e-0d7447fae907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:14.110937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 757/1504] 10.222.0.39 () {58 vars in 1061 bytes} [Sun Sep 4 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:10:14.118684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad5d9c-a2fc-44ee-a23c-5e0abade637d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:14.120224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-73ad5d9c-a2fc-44ee-a23c-5e0abade637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:14.170013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:14.224488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad5d9c-a2fc-44ee-a23c-5e0abade637d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:14.298602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:14.299203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:14.299477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:14.299744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:14.301313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:14.401288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:14.402747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:14.406601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:14.421039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 748/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:10:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:10:14.468177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372307a3-380c-4f1e-a7eb-cd111ecb7c31 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:14.469842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372307a3-380c-4f1e-a7eb-cd111ecb7c31 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:14.469842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372307a3-380c-4f1e-a7eb-cd111ecb7c31 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:14.470805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 758/1506] 10.222.0.39 () {52 vars in 830 bytes} [Sun Sep 4 16:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:10:14.474594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:14.475773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:14.574703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fab21b-cb5d-479a-8225-983c78fdc535 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:14.675014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:14.675647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:14.675928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:14.676203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] PATH_INFO: `/v3/users/3fbc34d0ca1e4a3d9608e01779b4e062` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:14.689364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbc34d0ca1e4a3d9608e01779b4e062)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:14.690418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:14.690418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Sep 04 16:10:14.690418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: Sep 04 16:10:14.691631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:14.701214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:14.856304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbc34d0ca1e4a3d9608e01779b4e062'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:14.856753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbc34d0ca1e4a3d9608e01779b4e062'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:14.904611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbc34d0ca1e4a3d9608e01779b4e062'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:14.905132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbc34d0ca1e4a3d9608e01779b4e062'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:14.929866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbc34d0ca1e4a3d9608e01779b4e062'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:14.948903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 749/1507] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:10:14 2022] DELETE /identity/v3/users/3fbc34d0ca1e4a3d9608e01779b4e062 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:10:14.953315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:14.953631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:14.953915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:15.011804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:10:15.013034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:15.087024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:15.099177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 759/1508] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:10:15.109066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:15.110252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:15.210977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280e8aa-2a19-4b02-9c6c-284854ee0e87 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:15.265918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:15.284324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fac1f-48ba-4d03-a6c0-e767132a4932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:15.285164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fac1f-48ba-4d03-a6c0-e767132a4932 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:15.285482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fac1f-48ba-4d03-a6c0-e767132a4932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:15.290481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fac1f-48ba-4d03-a6c0-e767132a4932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:15.290481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-166fac1f-48ba-4d03-a6c0-e767132a4932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:15.397410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fac1f-48ba-4d03-a6c0-e767132a4932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:15.398809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fac1f-48ba-4d03-a6c0-e767132a4932 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:15.419514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 750/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:10:15.867665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:10:15.868914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:10:15.880385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699a1db-ae61-4b6d-81fb-798888b83618 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:10:15.977455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:10:15.978114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:10:15.978388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:10:15.978651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] PATH_INFO: `/v3/projects/7323203d53dd486ba94de7cc575332ff` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:10:15.988938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7323203d53dd486ba94de7cc575332ff)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:10:15.989999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:10:15.989999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:10:15.989999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:10:15.991186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:10:16.106941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7323203d53dd486ba94de7cc575332ff', 'request_id': 'req-76d9893b-5ea2-487e-be01-bd78fb118c1a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:16.114906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7323203d53dd486ba94de7cc575332ff', 'request_id': 'req-76d9893b-5ea2-487e-be01-bd78fb118c1a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:16.128279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7323203d53dd486ba94de7cc575332ff', 'request_id': 'req-76d9893b-5ea2-487e-be01-bd78fb118c1a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:16.213423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] The token cache is being invalidate because project 7323203d53dd486ba94de7cc575332ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:10:16.213658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:16.214083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-76d9893b-5ea2-487e-be01-bd78fb118c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:10:16.215483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 760/1510] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:10:15 2022] DELETE /identity/v3/projects/7323203d53dd486ba94de7cc575332ff => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:10:25.417035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=62 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:10:26.211396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:11:40.508253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eaabb-1a2b-41bd-a9f3-c5dc97a87f58 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:40.508848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eaabb-1a2b-41bd-a9f3-c5dc97a87f58 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:40.508848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eaabb-1a2b-41bd-a9f3-c5dc97a87f58 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:40.510509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:11:40.515627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad949a-1d3b-41da-a281-a0504059d14b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:40.516045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad949a-1d3b-41da-a281-a0504059d14b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:40.516193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad949a-1d3b-41da-a281-a0504059d14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:40.598905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fbad949a-1d3b-41da-a281-a0504059d14b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:11:40.600280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fbad949a-1d3b-41da-a281-a0504059d14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:40.710297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:11:40.869968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eaabb-1a2b-41bd-a9f3-c5dc97a87f58 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:11:40.872097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-275eaabb-1a2b-41bd-a9f3-c5dc97a87f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:40.995498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eaabb-1a2b-41bd-a9f3-c5dc97a87f58 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:11:41.002033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61140937-50b7-4866-9858-b6ebce33549b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:11:41.002643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61140937-50b7-4866-9858-b6ebce33549b service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:41.002923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61140937-50b7-4866-9858-b6ebce33549b service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:41.003197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61140937-50b7-4866-9858-b6ebce33549b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:41.012071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61140937-50b7-4866-9858-b6ebce33549b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:11:41.014390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61140937-50b7-4866-9858-b6ebce33549b service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:11:41.023658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 752/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:11:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:11:41.854973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7980815-2325-4a20-9bca-b166d0d709b1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:41.855543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7980815-2325-4a20-9bca-b166d0d709b1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:41.855543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7980815-2325-4a20-9bca-b166d0d709b1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:41.856917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:11:41.861778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150ed35-5957-440a-8762-779ea7d35436 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:41.862201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150ed35-5957-440a-8762-779ea7d35436 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:41.862348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150ed35-5957-440a-8762-779ea7d35436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:41.882252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c150ed35-5957-440a-8762-779ea7d35436 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:11:41.883458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c150ed35-5957-440a-8762-779ea7d35436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:41.892329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:11:42.064474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7980815-2325-4a20-9bca-b166d0d709b1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:11:42.080223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7980815-2325-4a20-9bca-b166d0d709b1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:11:42.085759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7f987-ad9f-43a3-91a3-6cb60b541206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:11:42.086398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7f987-ad9f-43a3-91a3-6cb60b541206 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:42.086678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7f987-ad9f-43a3-91a3-6cb60b541206 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:42.086953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7f987-ad9f-43a3-91a3-6cb60b541206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:42.095693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7f987-ad9f-43a3-91a3-6cb60b541206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:11:42.097326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7f987-ad9f-43a3-91a3-6cb60b541206 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:11:42.106985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 763/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:11:42.928367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8325-6acb-4073-95a9-faa4d0781e0c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:42.928908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8325-6acb-4073-95a9-faa4d0781e0c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:42.929029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8325-6acb-4073-95a9-faa4d0781e0c None None] PATH_INFO: `/v3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:42.930261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 754/1517] 10.222.0.27 () {54 vars in 925 bytes} [Sun Sep 4 16:11:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:11:42.936176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbfdd1-52ff-4d60-8f11-66e79bf98973 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:42.936733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbfdd1-52ff-4d60-8f11-66e79bf98973 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:42.937122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbfdd1-52ff-4d60-8f11-66e79bf98973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:42.998005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-42bbfdd1-52ff-4d60-8f11-66e79bf98973 None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:11:42.999484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-42bbfdd1-52ff-4d60-8f11-66e79bf98973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:43.017231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:11:43.070712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 764/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Sep 4 16:11:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:11:47.037268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d407bc-5e72-4f37-ada2-04a00ca6b0a5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:47.037819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d407bc-5e72-4f37-ada2-04a00ca6b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:47.037819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d407bc-5e72-4f37-ada2-04a00ca6b0a5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:47.039086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 755/1519] 10.222.0.39 () {54 vars in 961 bytes} [Sun Sep 4 16:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:11:47.043947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b291a-2be7-40f9-b6ee-1cb16da6e18e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:47.044330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b291a-2be7-40f9-b6ee-1cb16da6e18e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:47.044505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b291a-2be7-40f9-b6ee-1cb16da6e18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:47.064863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a80b291a-2be7-40f9-b6ee-1cb16da6e18e None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:11:47.066091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a80b291a-2be7-40f9-b6ee-1cb16da6e18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:47.074505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 765/1520] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Sep 4 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:11:47.081886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d407bc-5e72-4f37-ada2-04a00ca6b0a5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:11:47.095694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d407bc-5e72-4f37-ada2-04a00ca6b0a5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:11:47.101140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8b56d-4f65-4f96-8e32-b351840aa7e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:11:47.101752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8b56d-4f65-4f96-8e32-b351840aa7e6 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:47.102032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8b56d-4f65-4f96-8e32-b351840aa7e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:47.102316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8b56d-4f65-4f96-8e32-b351840aa7e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:47.111306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8b56d-4f65-4f96-8e32-b351840aa7e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:11:47.112883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8b56d-4f65-4f96-8e32-b351840aa7e6 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:11:47.122324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 756/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:11:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:11:53.344621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa1fc-f6d3-42db-aed9-b2ef49a42ff2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:53.345120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa1fc-f6d3-42db-aed9-b2ef49a42ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:53.345227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fa1fc-f6d3-42db-aed9-b2ef49a42ff2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:53.346644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:11:53.351491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48e57-fc3b-4497-8ddc-976a8ba486ca None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:53.351965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48e57-fc3b-4497-8ddc-976a8ba486ca None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:53.352119 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48e57-fc3b-4497-8ddc-976a8ba486ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:53.371533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-edd48e57-fc3b-4497-8ddc-976a8ba486ca None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:11:53.372719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-edd48e57-fc3b-4497-8ddc-976a8ba486ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:53.382061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:11:53.541307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fa1fc-f6d3-42db-aed9-b2ef49a42ff2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:11:53.555740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fa1fc-f6d3-42db-aed9-b2ef49a42ff2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:11:53.561551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82be943-b88b-459c-8f41-de3d95d7381b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:11:53.562185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82be943-b88b-459c-8f41-de3d95d7381b service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:53.562473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82be943-b88b-459c-8f41-de3d95d7381b service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:53.562752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82be943-b88b-459c-8f41-de3d95d7381b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:53.570932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82be943-b88b-459c-8f41-de3d95d7381b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:11:53.576325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82be943-b88b-459c-8f41-de3d95d7381b service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:11:53.586021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 767/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:11:54.406185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cebd26-af20-41b1-83e7-388e06633e4c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:54.406705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cebd26-af20-41b1-83e7-388e06633e4c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:54.406814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cebd26-af20-41b1-83e7-388e06633e4c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:54.408118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 16:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:11:54.413516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1976a-818c-4af1-8c8c-00e16c1b2ddc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:54.413987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1976a-818c-4af1-8c8c-00e16c1b2ddc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:54.414098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1976a-818c-4af1-8c8c-00e16c1b2ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:54.433673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d5d1976a-818c-4af1-8c8c-00e16c1b2ddc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:11:54.434867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d5d1976a-818c-4af1-8c8c-00e16c1b2ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:54.443118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:11:54.609221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cebd26-af20-41b1-83e7-388e06633e4c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:11:54.623355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cebd26-af20-41b1-83e7-388e06633e4c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:11:54.629384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f05ee7-4556-4460-a1ef-fef3599d29b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:11:54.630004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05ee7-4556-4460-a1ef-fef3599d29b7 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:54.630284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05ee7-4556-4460-a1ef-fef3599d29b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:54.630552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05ee7-4556-4460-a1ef-fef3599d29b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:54.638788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f05ee7-4556-4460-a1ef-fef3599d29b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:11:54.641101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f05ee7-4556-4460-a1ef-fef3599d29b7 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:11:54.650325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 759/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:11:58.432244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa51375-7e36-4444-814f-6e0f7189babe None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:58.432765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa51375-7e36-4444-814f-6e0f7189babe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:58.432870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa51375-7e36-4444-814f-6e0f7189babe None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:58.434186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:11:58.439977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21f667-5a0b-44c1-a231-a1d7b578a664 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:58.440435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21f667-5a0b-44c1-a231-a1d7b578a664 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:58.440570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21f667-5a0b-44c1-a231-a1d7b578a664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:58.460219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6b21f667-5a0b-44c1-a231-a1d7b578a664 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:11:58.466394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6b21f667-5a0b-44c1-a231-a1d7b578a664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:58.469369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:11:58.757401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa51375-7e36-4444-814f-6e0f7189babe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:11:58.759154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5aa51375-7e36-4444-814f-6e0f7189babe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:11:58.837276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa51375-7e36-4444-814f-6e0f7189babe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:11:58.840892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadd3d4-3d9d-4be4-a72e-297adfb8ddb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:11:58.841490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadd3d4-3d9d-4be4-a72e-297adfb8ddb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:11:58.841800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadd3d4-3d9d-4be4-a72e-297adfb8ddb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:11:58.842076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadd3d4-3d9d-4be4-a72e-297adfb8ddb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:11:58.851794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadd3d4-3d9d-4be4-a72e-297adfb8ddb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:11:58.853234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadd3d4-3d9d-4be4-a72e-297adfb8ddb8 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:11:58.862525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 770/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:12:00.774305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45590290-d27f-45f7-9bba-2f85111f3fa9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:00.774831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45590290-d27f-45f7-9bba-2f85111f3fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:00.774831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45590290-d27f-45f7-9bba-2f85111f3fa9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:00.776210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:12:00.781980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1309-2414-424d-b729-a6e619161dd2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:00.782382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1309-2414-424d-b729-a6e619161dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:00.782541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1309-2414-424d-b729-a6e619161dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:00.802746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-51eb1309-2414-424d-b729-a6e619161dd2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:12:00.803957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-51eb1309-2414-424d-b729-a6e619161dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:12:00.811850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:12:01.099883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45590290-d27f-45f7-9bba-2f85111f3fa9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:12:01.111342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45590290-d27f-45f7-9bba-2f85111f3fa9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:12:01.114846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad12ad4-23ef-4826-b943-7dff6c5531e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:12:01.115442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad12ad4-23ef-4826-b943-7dff6c5531e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:01.115713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad12ad4-23ef-4826-b943-7dff6c5531e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:01.115980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad12ad4-23ef-4826-b943-7dff6c5531e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:01.125538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad12ad4-23ef-4826-b943-7dff6c5531e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:12:01.126887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad12ad4-23ef-4826-b943-7dff6c5531e0 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:12:01.136421 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 762/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:12:04.461496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0091a6-18eb-4aa2-8dd4-206f1686cc50 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:04.462069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0091a6-18eb-4aa2-8dd4-206f1686cc50 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:04.462166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0091a6-18eb-4aa2-8dd4-206f1686cc50 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:04.463582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:12:04.469609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad7c9-2da6-450d-90a1-2060065b8ea9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:04.470017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad7c9-2da6-450d-90a1-2060065b8ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:04.470200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad7c9-2da6-450d-90a1-2060065b8ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:04.490796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d02ad7c9-2da6-450d-90a1-2060065b8ea9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:12:04.491999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d02ad7c9-2da6-450d-90a1-2060065b8ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:12:04.499824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:12:05.055576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0091a6-18eb-4aa2-8dd4-206f1686cc50 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:12:05.057269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd0091a6-18eb-4aa2-8dd4-206f1686cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:12:05.122389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:12:05.135877 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0091a6-18eb-4aa2-8dd4-206f1686cc50 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:12:05.139527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e6dcd-3abe-4b9f-afb3-e5493d07706b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:12:05.147635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e6dcd-3abe-4b9f-afb3-e5493d07706b service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:05.147635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e6dcd-3abe-4b9f-afb3-e5493d07706b service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:05.147635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e6dcd-3abe-4b9f-afb3-e5493d07706b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:05.149527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e6dcd-3abe-4b9f-afb3-e5493d07706b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:12:05.150994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e6dcd-3abe-4b9f-afb3-e5493d07706b service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:12:05.160793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 773/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:12:07.175130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa3517-ebfa-4e30-a3ff-ccf1480d0f86 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:07.175695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa3517-ebfa-4e30-a3ff-ccf1480d0f86 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:07.175695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa3517-ebfa-4e30-a3ff-ccf1480d0f86 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:07.177031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 16:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:12:07.182235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5375-31ce-4489-8564-7a1067d8cd5a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:07.182641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5375-31ce-4489-8564-7a1067d8cd5a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:07.182793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5375-31ce-4489-8564-7a1067d8cd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:07.203352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4b6b5375-31ce-4489-8564-7a1067d8cd5a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:12:07.205167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4b6b5375-31ce-4489-8564-7a1067d8cd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:12:07.213148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:12:07.768477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa3517-ebfa-4e30-a3ff-ccf1480d0f86 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:12:07.770202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0eaa3517-ebfa-4e30-a3ff-ccf1480d0f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:12:07.777688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:12:07.784275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa3517-ebfa-4e30-a3ff-ccf1480d0f86 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:12:07.788635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bf96e-4d92-4c26-b9a2-732f0acf85ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:12:07.788635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bf96e-4d92-4c26-b9a2-732f0acf85ca service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:12:07.788635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bf96e-4d92-4c26-b9a2-732f0acf85ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:12:07.789074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bf96e-4d92-4c26-b9a2-732f0acf85ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:12:07.797941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bf96e-4d92-4c26-b9a2-732f0acf85ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:12:07.799361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bf96e-4d92-4c26-b9a2-732f0acf85ca service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:12:07.808605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 765/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:12:17.197817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=11 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:12:17.809985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:30.964923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e16ba6-e50e-4989-a0c5-c25334317f5e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:30.965594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e16ba6-e50e-4989-a0c5-c25334317f5e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:30.965594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e16ba6-e50e-4989-a0c5-c25334317f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:30.994443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-29e16ba6-e50e-4989-a0c5-c25334317f5e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:30.995690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29e16ba6-e50e-4989-a0c5-c25334317f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.009067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 775/1540] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.024278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf64ec9-eaa6-4c68-ae9a-d5c2bc6fe080 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.024878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf64ec9-eaa6-4c68-ae9a-d5c2bc6fe080 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.024878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf64ec9-eaa6-4c68-ae9a-d5c2bc6fe080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.036692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38aa5-6d9b-4b70-bdfc-435ffd992a8c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.037033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38aa5-6d9b-4b70-bdfc-435ffd992a8c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.037247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38aa5-6d9b-4b70-bdfc-435ffd992a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.059353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3bf64ec9-eaa6-4c68-ae9a-d5c2bc6fe080 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.060573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3bf64ec9-eaa6-4c68-ae9a-d5c2bc6fe080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.070583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-22f38aa5-6d9b-4b70-bdfc-435ffd992a8c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.071800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-22f38aa5-6d9b-4b70-bdfc-435ffd992a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.073077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 766/1541] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:31.078651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf64ec9-eaa6-4c68-ae9a-d5c2bc6fe080 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.088760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 776/1542] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:31.097195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf64ec9-eaa6-4c68-ae9a-d5c2bc6fe080 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.098495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b00d42-97d0-4226-8fbb-03c5346339a6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.098782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b00d42-97d0-4226-8fbb-03c5346339a6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.099062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b00d42-97d0-4226-8fbb-03c5346339a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.105359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.106104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.106391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.106912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.107576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.107880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.108892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:13:31.108892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Sep 04 16:13:31.108892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: Sep 04 16:13:31.114145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd0c5f-f2ad-4180-abcf-2a4b19156709 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.136537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 767/1543] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:13:31.137662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-69b00d42-97d0-4226-8fbb-03c5346339a6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.138971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-69b00d42-97d0-4226-8fbb-03c5346339a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.155953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 777/1544] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:31.156769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10352598-4d65-4649-8d5d-cce945218e25 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.157108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10352598-4d65-4649-8d5d-cce945218e25 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.157574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10352598-4d65-4649-8d5d-cce945218e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.161075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b00d42-97d0-4226-8fbb-03c5346339a6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.185926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b00d42-97d0-4226-8fbb-03c5346339a6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.190082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-10352598-4d65-4649-8d5d-cce945218e25 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.191279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-10352598-4d65-4649-8d5d-cce945218e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.199011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4114da-b01e-452f-bf8c-848aa8d087d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.200610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4114da-b01e-452f-bf8c-848aa8d087d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.200901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4114da-b01e-452f-bf8c-848aa8d087d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.201729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4114da-b01e-452f-bf8c-848aa8d087d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.202917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4114da-b01e-452f-bf8c-848aa8d087d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.204810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-3e4114da-b01e-452f-bf8c-848aa8d087d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:13:31.204810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:13:31.204810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:13:31.206520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4114da-b01e-452f-bf8c-848aa8d087d4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.213890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 768/1545] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:31.215250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d733a-8764-4d06-8160-95a03adade35 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.215484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d733a-8764-4d06-8160-95a03adade35 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.215756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d733a-8764-4d06-8160-95a03adade35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.254869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a40d733a-8764-4d06-8160-95a03adade35 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.256076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a40d733a-8764-4d06-8160-95a03adade35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.271199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 769/1546] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.272094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d733a-8764-4d06-8160-95a03adade35 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.282461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d733a-8764-4d06-8160-95a03adade35 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.287087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd974f9-d57d-4bda-b1ff-e0af615c1155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.287686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd974f9-d57d-4bda-b1ff-e0af615c1155 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.287965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd974f9-d57d-4bda-b1ff-e0af615c1155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.288234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd974f9-d57d-4bda-b1ff-e0af615c1155 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.288876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd974f9-d57d-4bda-b1ff-e0af615c1155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.289174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd974f9-d57d-4bda-b1ff-e0af615c1155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.290799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd974f9-d57d-4bda-b1ff-e0af615c1155 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.303272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 770/1547] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:31.304384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b31db-103a-4991-8030-583e0bc6c51d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.304666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b31db-103a-4991-8030-583e0bc6c51d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.304939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b31db-103a-4991-8030-583e0bc6c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.324662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3e9b31db-103a-4991-8030-583e0bc6c51d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.325873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3e9b31db-103a-4991-8030-583e0bc6c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.334614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 771/1548] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.335207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96b320-1086-4740-babe-ca765d8c589f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.335489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96b320-1086-4740-babe-ca765d8c589f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.335760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96b320-1086-4740-babe-ca765d8c589f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.350843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 778/1549] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:31.350843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679952-bfb7-465e-9cd8-a6e22775539e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.350843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679952-bfb7-465e-9cd8-a6e22775539e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.350843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679952-bfb7-465e-9cd8-a6e22775539e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.357676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ad96b320-1086-4740-babe-ca765d8c589f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.358861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ad96b320-1086-4740-babe-ca765d8c589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.370251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d5679952-bfb7-465e-9cd8-a6e22775539e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.371444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d5679952-bfb7-465e-9cd8-a6e22775539e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.420043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 772/1550] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.421294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68684c-9cdb-4286-aee1-e0f96db0570e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.421588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68684c-9cdb-4286-aee1-e0f96db0570e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.421934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68684c-9cdb-4286-aee1-e0f96db0570e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.430003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 779/1551] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.431472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736a4fb-f24e-4ef4-a881-00e24ee2a61b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.431757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736a4fb-f24e-4ef4-a881-00e24ee2a61b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.432026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736a4fb-f24e-4ef4-a881-00e24ee2a61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9b68684c-9cdb-4286-aee1-e0f96db0570e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.448582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9b68684c-9cdb-4286-aee1-e0f96db0570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.451962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9736a4fb-f24e-4ef4-a881-00e24ee2a61b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.454073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9736a4fb-f24e-4ef4-a881-00e24ee2a61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.457299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 773/1552] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.458131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68684c-9cdb-4286-aee1-e0f96db0570e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.465811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 780/1553] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.465811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fd6b7-c559-4322-9bf8-82e9a9c1e04a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.465811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fd6b7-c559-4322-9bf8-82e9a9c1e04a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.465811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fd6b7-c559-4322-9bf8-82e9a9c1e04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.469483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68684c-9cdb-4286-aee1-e0f96db0570e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.474430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.475025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.475443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.475708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.476328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.478253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.483176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5c9fd6b7-c559-4322-9bf8-82e9a9c1e04a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.484412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5c9fd6b7-c559-4322-9bf8-82e9a9c1e04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.492857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 781/1554] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.494157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cc562-be70-4ba1-b9ab-d516a32d499f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.494426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cc562-be70-4ba1-b9ab-d516a32d499f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.494837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cc562-be70-4ba1-b9ab-d516a32d499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.517289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-100cc562-be70-4ba1-b9ab-d516a32d499f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.518526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-100cc562-be70-4ba1-b9ab-d516a32d499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.526917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 782/1555] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.527834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cc562-be70-4ba1-b9ab-d516a32d499f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.538131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cc562-be70-4ba1-b9ab-d516a32d499f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.542655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cd069-1f7b-41c2-ac7c-26912d15db09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.543237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd069-1f7b-41c2-ac7c-26912d15db09 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.543517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd069-1f7b-41c2-ac7c-26912d15db09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.543791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd069-1f7b-41c2-ac7c-26912d15db09 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.544415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cd069-1f7b-41c2-ac7c-26912d15db09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.546369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cd069-1f7b-41c2-ac7c-26912d15db09 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.633408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 774/1556] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:31.633408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.657172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 783/1557] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:31.658477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0342124-642b-4fe7-9e96-5dfe380631bd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.662937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0342124-642b-4fe7-9e96-5dfe380631bd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.662937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0342124-642b-4fe7-9e96-5dfe380631bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.678940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b0342124-642b-4fe7-9e96-5dfe380631bd None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.680147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b0342124-642b-4fe7-9e96-5dfe380631bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.702702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd68fd-4aab-4817-8d0f-be28002235ef admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.710352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8258a1f-e6cc-40fb-8323-4d680ae2cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.711303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8258a1f-e6cc-40fb-8323-4d680ae2cbfd admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.711768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8258a1f-e6cc-40fb-8323-4d680ae2cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.712230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8258a1f-e6cc-40fb-8323-4d680ae2cbfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.713311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8258a1f-e6cc-40fb-8323-4d680ae2cbfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.713849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8258a1f-e6cc-40fb-8323-4d680ae2cbfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.716574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8258a1f-e6cc-40fb-8323-4d680ae2cbfd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.732974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 775/1558] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:31.735581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5da29c-ebbb-432e-8597-a94012a96b71 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.735974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5da29c-ebbb-432e-8597-a94012a96b71 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.736349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5da29c-ebbb-432e-8597-a94012a96b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.756811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3d5da29c-ebbb-432e-8597-a94012a96b71 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.758063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3d5da29c-ebbb-432e-8597-a94012a96b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.765509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 784/1559] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.766491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0342124-642b-4fe7-9e96-5dfe380631bd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.767040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 776/1560] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.767040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62e1b0-6c5c-4af2-a5a3-617d0ea0e273 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.767040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62e1b0-6c5c-4af2-a5a3-617d0ea0e273 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.767040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62e1b0-6c5c-4af2-a5a3-617d0ea0e273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.775905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0342124-642b-4fe7-9e96-5dfe380631bd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.780600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.781348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.781794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.782067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.782709 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.783018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.783391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:31.784639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.788823 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6c62e1b0-6c5c-4af2-a5a3-617d0ea0e273 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:31.792679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6c62e1b0-6c5c-4af2-a5a3-617d0ea0e273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:31.793355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:31.796486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 785/1561] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:31.797944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.802944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 777/1562] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:31.802944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62e1b0-6c5c-4af2-a5a3-617d0ea0e273 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.814355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869c032-0195-4a43-872d-71644bc0e5f8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.815723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62e1b0-6c5c-4af2-a5a3-617d0ea0e273 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.819041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.821367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.821367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.821367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.821704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.821820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.821924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.822012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.822099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.822099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.822295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.822718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.823281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:13:31.823281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Sep 04 16:13:31.823281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: Sep 04 16:13:31.824484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.835046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 786/1563] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:31.835973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.846182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbeb9b-2627-4f57-80c9-c665c9e2bc70 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.850989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.851581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.851872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.852180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.852817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.854457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.864655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 787/1564] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:31.865414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.874404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f8926-d8b4-4cb2-a9cb-896fbbeba83d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.879716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.880273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.880549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.880818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.881418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.883299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.900357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 778/1565] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:31.900357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.908235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a596-2134-4ab8-aa87-14076985c8d0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.913501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.914120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.914401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.924024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.924024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.924024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.924024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.930007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 779/1566] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:31.930893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.941057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d545366-25fa-4611-ae68-da5c9d72681c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.945627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.946206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.946501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.946765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.947395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.947697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.949257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.960353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 780/1567] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:31.961255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.970841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce3ab9-2057-4786-a56f-7c7f659bad4a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:31.975451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:31.976059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:31.976343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:31.976618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:31.977254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:31.977583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:31.981464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 788/1568] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:31.981960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:31.982079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:31.993377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 781/1569] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:31.994061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.009597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bdf0b-9c29-4ee6-be23-be3be6a0cf49 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.047628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056c8c0-2882-4f0b-ada3-15f34a67a459 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.052354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.052967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.053250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.053526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.054191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.056109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.071071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.071690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.071974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.072280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.072913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.074889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.133348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 789/1570] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.134221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.157428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 782/1571] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:31 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.157428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.197051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1182fe5-9137-4fa3-a509-0c292fdea061 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.220646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f758e2-b8fd-4f49-90a7-5077c0b3a222 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.225496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.226084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.226369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.226646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.227290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.228914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.239794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 783/1572] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:32.240622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.241914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:32.250164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120630-2869-4611-ad2d-4996f51ea846 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.254911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.256996 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.257129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.257227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.257326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.257425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.257523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.257639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.257747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.257837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.258506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.258622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.269063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 790/1573] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:32.269729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.279123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f189c0-8eb6-476d-80c8-b3013b610e44 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.288407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.288407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.288407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.288407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.288815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.288815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.372849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 791/1574] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.373418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 784/1575] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.373804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.374648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.379584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:32.440014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430f657-6cd6-407a-8657-9e3358ddc104 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.441175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d45760-1a98-47cd-93e0-f493d427f620 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.445337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.445774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.446016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.448909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.449034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.449124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.449200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.449200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.449449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.449550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.449673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.449771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.542952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 792/1576] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.543711 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.553696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a546d-4f5e-480d-99ec-504a499ec18b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.558352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.558927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.559205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.559479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.560084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.562026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.565265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 785/1577] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.569669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.629534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3f90a-b9e0-43d0-b4dd-868f532ecf8c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.634418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.635023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.635310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.635585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.636475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.638516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.651608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 793/1578] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.652451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.662663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ab6ca-8614-4573-9376-2053b05a88df admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.667679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.668264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.673355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.673355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.673355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.673355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.682215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 794/1579] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:32.682973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.692480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba31135-4a6a-41ea-8ab1-374e58d01135 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.697149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.697802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.698089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.698367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] PATH_INFO: `/v3/projects/3219dc7400ec4957a31b1b0fabfd8c56/users/1623e93ad6844cacae7cc1a71b339f22/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.712980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3219dc7400ec4957a31b1b0fabfd8c56, user_id=1623e93ad6844cacae7cc1a71b339f22, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.714047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:13:32.714047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:13:32.714047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:13:32.716047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.717595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 786/1580] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.718553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.728260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98558960-d6a0-4057-baa7-6c4309e40111 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.732844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.733434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.733749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.734023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.734644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.736779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.772206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 795/1581] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:32 2022] PUT /identity/v3/projects/3219dc7400ec4957a31b1b0fabfd8c56/users/1623e93ad6844cacae7cc1a71b339f22/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:32.773028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.811425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 787/1582] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/users => generated 279 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.811425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.822013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d9253-a619-4470-91f1-e2e8bb2d2083 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.832548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:32.836370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5f78-0d84-4594-b6a0-5c4803dd19f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.841115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.845872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.845872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.845872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.845872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.845872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.880322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.880926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.881284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.881555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.882213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.883793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.885008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:32.894410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 788/1583] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:32.895266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.904978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a404269-a526-45af-ab3f-899e23627c8a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.910533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.911097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.911375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.911644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.912249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.914139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.965523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 796/1584] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/users => generated 286 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.966383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.976382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008db57f-30c2-43ff-a4a5-c7a5974ecd6e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:32.981047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:32.981670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:32.982228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:32.982228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:32.982841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:32.984430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:32.994915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 789/1585] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:32 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:32.998177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 797/1586] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:32.998316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:32.998413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.006594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521263-9632-4197-a624-a38548d2d4c6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.007436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0466db7-d74f-43b8-adbd-40342c2b9798 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.011560 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efb13f-032f-45fa-8d79-f1f0444e4c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.012147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efb13f-032f-45fa-8d79-f1f0444e4c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.012259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.012430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efb13f-032f-45fa-8d79-f1f0444e4c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.012702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efb13f-032f-45fa-8d79-f1f0444e4c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.013336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efb13f-032f-45fa-8d79-f1f0444e4c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.014378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.014666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.014947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] PATH_INFO: `/v3/projects/f7786c29c31642a4a02db494e8a57e59/users/dd48767bd3fa451fbeb5bbf40084d6b0/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.015058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efb13f-032f-45fa-8d79-f1f0444e4c11 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.026245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 798/1587] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.027470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fae9d-9b39-4beb-9e84-f6f257f816ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.027772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fae9d-9b39-4beb-9e84-f6f257f816ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.028028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fae9d-9b39-4beb-9e84-f6f257f816ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.031611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7786c29c31642a4a02db494e8a57e59, user_id=dd48767bd3fa451fbeb5bbf40084d6b0, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.035707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.047190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7c6fae9d-9b39-4beb-9e84-f6f257f816ad None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:33.049037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7c6fae9d-9b39-4beb-9e84-f6f257f816ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:33.056622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 799/1588] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:33.057426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fae9d-9b39-4beb-9e84-f6f257f816ad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.068087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fae9d-9b39-4beb-9e84-f6f257f816ad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.073120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.073752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.074034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.074311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.074952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.076552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.087142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 800/1589] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.093030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.098674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a8e09-780d-486f-a4e1-a2d8cd2baa6f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.103208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.103759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.104071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.104357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.104952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.106520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.118606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 801/1590] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.118606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.123731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 790/1591] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:32 2022] PUT /identity/v3/projects/f7786c29c31642a4a02db494e8a57e59/users/dd48767bd3fa451fbeb5bbf40084d6b0/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:33.123731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.130475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b810b1-d2d2-4174-9e9b-c775737f7a37 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.190143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.190762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.191047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.191326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.191975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.193640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.193961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a8f2f-5568-42ba-8bfd-a7dd089ee3dc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.199227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.199576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.199746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.200004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.200645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.202318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.203755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 802/1592] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.205221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.214811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69579cd-6b86-46ed-89b4-485b1def1619 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.215873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 791/1593] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.216762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.223089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.223689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.223974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.224254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.224885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.226527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.226974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2d8f9-becf-4401-b6e7-cb4b1927be60 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.231452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.232027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.232304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.232573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.233191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.234778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.238486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 803/1594] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.239290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.240943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:33.249706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 792/1595] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.249706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.256689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f92b4-2440-4a40-8a7c-9670e3a4af52 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.261182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.262038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.262319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.262844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.263469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.265133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.275881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 793/1596] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.276757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.286951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5928e4-5d25-4f3e-a35f-7789a0396410 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.291561 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.292195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.292478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.292754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] PATH_INFO: `/v3/projects/17338e3b6b204595a43fddc1c00ed26b/users/f1af03ad31a34f319763a6b5451ae26e/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.305789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63664-3004-4d23-a29b-786d3ae71330 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.307230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17338e3b6b204595a43fddc1c00ed26b, user_id=f1af03ad31a34f319763a6b5451ae26e, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.309367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99562b5b-61da-4526-a374-182b5774f78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.309905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.310046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99562b5b-61da-4526-a374-182b5774f78b service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.310273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99562b5b-61da-4526-a374-182b5774f78b service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.310542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99562b5b-61da-4526-a374-182b5774f78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.320103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99562b5b-61da-4526-a374-182b5774f78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.321475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99562b5b-61da-4526-a374-182b5774f78b service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.331763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 804/1597] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:33.332976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c7294-85a5-4bb1-8c42-722543c9b1af None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.333264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c7294-85a5-4bb1-8c42-722543c9b1af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.333541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c7294-85a5-4bb1-8c42-722543c9b1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.364179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bc4c7294-85a5-4bb1-8c42-722543c9b1af None None] MFA Rules not processed for user `dd48767bd3fa451fbeb5bbf40084d6b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:33.365399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bc4c7294-85a5-4bb1-8c42-722543c9b1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:33.376988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:33.385855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 794/1598] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:33 2022] PUT /identity/v3/projects/17338e3b6b204595a43fddc1c00ed26b/users/f1af03ad31a34f319763a6b5451ae26e/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:33.388676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.430989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 805/1599] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:33.434482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c7294-85a5-4bb1-8c42-722543c9b1af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.445657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c7294-85a5-4bb1-8c42-722543c9b1af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.458539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abab0-2cbd-48ee-9bbb-78c45dd6303e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.463816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.464497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.464785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.465165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.465768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.467943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.475981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:33.481373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 795/1600] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.485698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.496273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734141-9012-4c17-b15e-b17cf9ea58f4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.502292 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.502963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.503244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.503520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.504217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.505990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.516250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.517092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.517415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.517726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] PATH_INFO: `/v3/projects/cb043a0ecad3466bbd5a5874126d2810/users/a9d55c52717546b09d93d956a29f49a9/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.522588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 796/1601] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:33.523592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.535216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1684-efcb-404d-8558-5b5fd03d565a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.536935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb043a0ecad3466bbd5a5874126d2810, user_id=a9d55c52717546b09d93d956a29f49a9, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.539777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.540392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2621e-9ef9-45b6-b648-0332acd435cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.541189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2621e-9ef9-45b6-b648-0332acd435cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.541535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2621e-9ef9-45b6-b648-0332acd435cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.541931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2621e-9ef9-45b6-b648-0332acd435cf admin admin] PATH_INFO: `/v3/projects/183fb107080444bf99d0369dde5e5466/users/5f8aa81e13da42a8bb35f8a983033a91/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.558084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2621e-9ef9-45b6-b648-0332acd435cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=183fb107080444bf99d0369dde5e5466, user_id=5f8aa81e13da42a8bb35f8a983033a91, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.561201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2621e-9ef9-45b6-b648-0332acd435cf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.608601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 806/1602] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:33 2022] PUT /identity/v3/projects/cb043a0ecad3466bbd5a5874126d2810/users/a9d55c52717546b09d93d956a29f49a9/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:33.611268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.631998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 797/1603] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:33 2022] PUT /identity/v3/projects/183fb107080444bf99d0369dde5e5466/users/5f8aa81e13da42a8bb35f8a983033a91/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:33.634592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c04d-af1e-4561-b224-a51ada9db946 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.634592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c04d-af1e-4561-b224-a51ada9db946 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.634592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c04d-af1e-4561-b224-a51ada9db946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.658645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9ac5c04d-af1e-4561-b224-a51ada9db946 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:33.659884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9ac5c04d-af1e-4561-b224-a51ada9db946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:33.702951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a80a3-3eb6-4625-b73e-5b57fcadcfa8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.738520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 798/1604] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:33.740092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0588-fad4-427c-898a-be178afb8fce None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.740569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0588-fad4-427c-898a-be178afb8fce None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.741041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0588-fad4-427c-898a-be178afb8fce None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.743348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 799/1605] 10.222.0.39 () {54 vars in 962 bytes} [Sun Sep 4 16:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:13:33.744337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb0588-fad4-427c-898a-be178afb8fce None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.757950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb0588-fad4-427c-898a-be178afb8fce None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:33.763092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.763741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.764019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.764287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] PATH_INFO: `/v3/projects/2f68c2eedf0b4c52ba46d302f28c92ee/users/35a0b70ddb65478194fa600e92018265/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.771013 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce65f54-e31b-4168-9080-22c4a8623622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:33.771675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce65f54-e31b-4168-9080-22c4a8623622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.771955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce65f54-e31b-4168-9080-22c4a8623622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.772405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce65f54-e31b-4168-9080-22c4a8623622 admin admin] PATH_INFO: `/v3/projects/328290a36e0f451c9117bcce81c7b753/users/9e563e3bbd07468f9c7a29837a10de0c/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.780098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f68c2eedf0b4c52ba46d302f28c92ee, user_id=35a0b70ddb65478194fa600e92018265, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.783447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.787069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce65f54-e31b-4168-9080-22c4a8623622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=328290a36e0f451c9117bcce81c7b753, user_id=9e563e3bbd07468f9c7a29837a10de0c, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:33.789826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce65f54-e31b-4168-9080-22c4a8623622 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:33.900362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 800/1606] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:33 2022] PUT /identity/v3/projects/2f68c2eedf0b4c52ba46d302f28c92ee/users/35a0b70ddb65478194fa600e92018265/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:33.904262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 807/1607] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:33 2022] PUT /identity/v3/projects/328290a36e0f451c9117bcce81c7b753/users/9e563e3bbd07468f9c7a29837a10de0c/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:33.904476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:33.904630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838893-cf34-45c1-beea-74757e9a1d7d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:33.904630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838893-cf34-45c1-beea-74757e9a1d7d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:33.904630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838893-cf34-45c1-beea-74757e9a1d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:33.930863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4a838893-cf34-45c1-beea-74757e9a1d7d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:33.934114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4a838893-cf34-45c1-beea-74757e9a1d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:33.990880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4617af-fad9-4b59-927e-e99b7eef9c32 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.002131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 808/1608] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:34.003474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eeb1d1-6863-40c6-b6a8-f31cdfc5a85a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.004150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eeb1d1-6863-40c6-b6a8-f31cdfc5a85a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.004150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eeb1d1-6863-40c6-b6a8-f31cdfc5a85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.033924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-45eeb1d1-6863-40c6-b6a8-f31cdfc5a85a None None] MFA Rules not processed for user `5f8aa81e13da42a8bb35f8a983033a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:34.035718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-45eeb1d1-6863-40c6-b6a8-f31cdfc5a85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:34.051669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.052379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.052800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.053081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] PATH_INFO: `/v3/projects/1cc5ea4158344d5c8a35bc5783410620/users/6e5c22b631df4600b9f920120b67fc67/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.053841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:34.068526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc5ea4158344d5c8a35bc5783410620, user_id=6e5c22b631df4600b9f920120b67fc67, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.071697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.100862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:34.109966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 809/1609] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:34.111374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461e723-a7ed-471f-a6e7-fec901b70dcf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.111658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461e723-a7ed-471f-a6e7-fec901b70dcf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.112362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461e723-a7ed-471f-a6e7-fec901b70dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.134531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 801/1610] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:33 2022] PUT /identity/v3/projects/1cc5ea4158344d5c8a35bc5783410620/users/6e5c22b631df4600b9f920120b67fc67/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:34.134531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.155075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a461e723-a7ed-471f-a6e7-fec901b70dcf None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:34.156554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a461e723-a7ed-471f-a6e7-fec901b70dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:34.193801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412388c-7546-4d41-a90a-d7200ea8defc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.197487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b6839-3a18-4f17-82b2-9699fcbf1206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.198153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b6839-3a18-4f17-82b2-9699fcbf1206 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.198756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b6839-3a18-4f17-82b2-9699fcbf1206 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.198885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b6839-3a18-4f17-82b2-9699fcbf1206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.228174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 810/1611] 10.222.0.39 () {58 vars in 1059 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:34.232115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebdde0-47c2-4a03-a9d4-71415a3cda74 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.232115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebdde0-47c2-4a03-a9d4-71415a3cda74 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.232115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebdde0-47c2-4a03-a9d4-71415a3cda74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.259177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-00ebdde0-47c2-4a03-a9d4-71415a3cda74 None None] MFA Rules not processed for user `35a0b70ddb65478194fa600e92018265`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:34.260377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-00ebdde0-47c2-4a03-a9d4-71415a3cda74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:34.264482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b6839-3a18-4f17-82b2-9699fcbf1206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.266039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b6839-3a18-4f17-82b2-9699fcbf1206 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.276547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 802/1612] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:34.277951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353781e-d449-4c61-9a4b-f4813f53bfad None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.278244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353781e-d449-4c61-9a4b-f4813f53bfad None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.278523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353781e-d449-4c61-9a4b-f4813f53bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.327852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0353781e-d449-4c61-9a4b-f4813f53bfad None None] MFA Rules not processed for user `9e563e3bbd07468f9c7a29837a10de0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:34.329476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0353781e-d449-4c61-9a4b-f4813f53bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:34.334745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 811/1613] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:34.336491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebdde0-47c2-4a03-a9d4-71415a3cda74 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.348897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebdde0-47c2-4a03-a9d4-71415a3cda74 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.352939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.353683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.354040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.354390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.365713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.367171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.380987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:34.381378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 812/1614] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:34.382376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.394597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7136d78-85c5-4a21-a1a0-05f52cb9c9f5 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.399779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.400444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.400734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.401068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.401761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.403797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.416369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 803/1615] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:34.418681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0e30f-3a63-4c5e-8399-9a5ac82510bb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.419327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0e30f-3a63-4c5e-8399-9a5ac82510bb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.419327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0e30f-3a63-4c5e-8399-9a5ac82510bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.443376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f3c0e30f-3a63-4c5e-8399-9a5ac82510bb None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:34.444741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f3c0e30f-3a63-4c5e-8399-9a5ac82510bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:34.452699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 804/1616] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:34.453585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0e30f-3a63-4c5e-8399-9a5ac82510bb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.463131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:34.467581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0e30f-3a63-4c5e-8399-9a5ac82510bb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.474196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.474821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.475101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.475379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.501792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 813/1617] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:34.501792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.544469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.545877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.556509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 805/1618] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:34.557412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.567875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974c70d-2cad-432f-9f6f-0bddd2364458 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.570686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb3baa-0240-4d88-a206-d7513de2cd8d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.575704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.575704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.575704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.575704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.576089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.576371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.589114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.589781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.590066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.590479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.591133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.596905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.716641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 806/1619] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:34.717691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.728136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 814/1620] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/projects => generated 373 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:34.729242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.803956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f1be3-92c6-41bd-96fd-68e08b3224fe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.877721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a27b9d-d4be-48ed-81e7-c97856640468 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.882658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:34.893785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.894532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.894892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.895326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.897317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.899919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.900538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.900824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.901100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.903562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.917505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.919776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:34.940807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 807/1621] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:34.942264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:34.953601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea23d4-a224-49ba-8150-30b96ac92dd9 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:34.960300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc1ce6-5c6b-4336-8242-2bb79321500c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:34.960908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc1ce6-5c6b-4336-8242-2bb79321500c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:34.961185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc1ce6-5c6b-4336-8242-2bb79321500c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:34.962117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc1ce6-5c6b-4336-8242-2bb79321500c admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:34.962956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc1ce6-5c6b-4336-8242-2bb79321500c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:34.965395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc1ce6-5c6b-4336-8242-2bb79321500c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.053162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 815/1622] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/users => generated 289 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:35.053162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.069064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4693c7-c9e1-4f5b-b269-09ab2b5e7697 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.075091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd17aae-b9c4-47e8-9abc-f9fff6a6b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.075718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd17aae-b9c4-47e8-9abc-f9fff6a6b0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.076006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd17aae-b9c4-47e8-9abc-f9fff6a6b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.076409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd17aae-b9c4-47e8-9abc-f9fff6a6b0c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.077236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd17aae-b9c4-47e8-9abc-f9fff6a6b0c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.079908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd17aae-b9c4-47e8-9abc-f9fff6a6b0c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.101530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 808/1623] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:34 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:35.103139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706055a-1d88-4f66-8daa-4bc85070c957 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.103525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706055a-1d88-4f66-8daa-4bc85070c957 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.104314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706055a-1d88-4f66-8daa-4bc85070c957 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.109744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 809/1624] 10.222.0.39 () {54 vars in 912 bytes} [Sun Sep 4 16:13:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:13:35.110648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706055a-1d88-4f66-8daa-4bc85070c957 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.125341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706055a-1d88-4f66-8daa-4bc85070c957 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.130514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.131120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.131423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.131690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.132321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.133958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.146076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 810/1625] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:35.146934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.158827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ec13f-bd07-4ff0-a45c-cb78e14e5500 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.163993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.164711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.165032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.165406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.166127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.167904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.171574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 816/1626] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:35 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:35.178329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ca7fb-979c-4d2f-b39e-9ab86444f8fb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.178329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ca7fb-979c-4d2f-b39e-9ab86444f8fb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.178329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ca7fb-979c-4d2f-b39e-9ab86444f8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.179109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:35.186911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 811/1627] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:35.187850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.195178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-055ca7fb-979c-4d2f-b39e-9ab86444f8fb None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:35.196586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-055ca7fb-979c-4d2f-b39e-9ab86444f8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:35.202953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8786c51-826b-43e8-a500-54c09cbe89da admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.210149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.216640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.216640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.216640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] PATH_INFO: `/v3/projects/89173a8fed6b4e6381a006abae66baa7/users/184ae94126674ddeae0aa365f92831ce/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.230263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=89173a8fed6b4e6381a006abae66baa7, user_id=184ae94126674ddeae0aa365f92831ce, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.233020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.281203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 817/1628] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:35.282478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ca7fb-979c-4d2f-b39e-9ab86444f8fb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.293635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ca7fb-979c-4d2f-b39e-9ab86444f8fb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.298895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.299504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.299839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.300132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.300684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.302329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.312611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 818/1629] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:35.315508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.323828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea1e40-7335-4aa3-a903-80a719b17a2b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.328463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.329761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.330040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.330319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] PATH_INFO: `/v3/projects/d5679ab32e2746c399563a107655d4e8/users/f7fbe9450cc445abb13d2e7ca654a099/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.340238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 812/1630] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:35 2022] PUT /identity/v3/projects/89173a8fed6b4e6381a006abae66baa7/users/184ae94126674ddeae0aa365f92831ce/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:35.344185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.352316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5679ab32e2746c399563a107655d4e8, user_id=f7fbe9450cc445abb13d2e7ca654a099, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.355675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.429686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851328-cf6f-4daf-b35c-2b285374f2bb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.435958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.435958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.435958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.436289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.466732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 819/1631] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:35 2022] PUT /identity/v3/projects/d5679ab32e2746c399563a107655d4e8/users/f7fbe9450cc445abb13d2e7ca654a099/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:35.466732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.560158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:35.568457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.570164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.599246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 813/1632] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:35.601802 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.637368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f62d10-34dc-4044-8bdb-e12bd5f13007 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.654050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.655479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.655479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.655732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] PATH_INFO: `/v3/projects/ce58f0db51124e5a805f8e273d2dde40/users/72a5a6ac38e44c558345315a4ec50a1c/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.692635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce58f0db51124e5a805f8e273d2dde40, user_id=72a5a6ac38e44c558345315a4ec50a1c, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.695851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.734465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d0dd8-833a-4aeb-9a2c-bd6d995cc3ac service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.740549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.744499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.744499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.744499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.744499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.744499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.767601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 814/1633] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:35.769309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.784244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c47683-a480-4969-b107-0f349541fec4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.789895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.790805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.791186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.792404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.793169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.795733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.812175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 815/1634] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:35.812175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.822165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 820/1635] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:35 2022] PUT /identity/v3/projects/ce58f0db51124e5a805f8e273d2dde40/users/72a5a6ac38e44c558345315a4ec50a1c/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:35.826050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:35.842033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:35.854631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567192c1-f8f9-4358-8c06-0f71bb6f17df admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.975899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bf144-cf51-47df-89e5-bec8e3d5679c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:35.984961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.986256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.986633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.987035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] PATH_INFO: `/v3/projects/89173a8fed6b4e6381a006abae66baa7/users/184ae94126674ddeae0aa365f92831ce/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.987149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032cf80-d288-49a4-914a-e2c8bd226712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:35.989662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89173a8fed6b4e6381a006abae66baa7, user_id=184ae94126674ddeae0aa365f92831ce, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:35.990918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032cf80-d288-49a4-914a-e2c8bd226712 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:35.991231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032cf80-d288-49a4-914a-e2c8bd226712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:35.991521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032cf80-d288-49a4-914a-e2c8bd226712 admin admin] PATH_INFO: `/v3/projects/d5679ab32e2746c399563a107655d4e8/users/f7fbe9450cc445abb13d2e7ca654a099/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:35.993059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:35.999202 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032cf80-d288-49a4-914a-e2c8bd226712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5679ab32e2746c399563a107655d4e8, user_id=f7fbe9450cc445abb13d2e7ca654a099, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:36.003125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032cf80-d288-49a4-914a-e2c8bd226712 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:36.017495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 816/1636] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:35 2022] PUT /identity/v3/projects/89173a8fed6b4e6381a006abae66baa7/users/184ae94126674ddeae0aa365f92831ce/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:36.018363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:36.046506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 821/1637] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:35 2022] PUT /identity/v3/projects/d5679ab32e2746c399563a107655d4e8/users/f7fbe9450cc445abb13d2e7ca654a099/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:36.046506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab571cbf-cde2-4ef9-adc6-0eb7e527aec6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:36.046506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab571cbf-cde2-4ef9-adc6-0eb7e527aec6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:36.046506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab571cbf-cde2-4ef9-adc6-0eb7e527aec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:36.084828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ab571cbf-cde2-4ef9-adc6-0eb7e527aec6 None None] MFA Rules not processed for user `184ae94126674ddeae0aa365f92831ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:36.086326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ab571cbf-cde2-4ef9-adc6-0eb7e527aec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:36.124562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fc917-b6ea-4c34-825e-a2ccdda56274 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:36.283414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 822/1638] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:36.283414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef88b0b-260c-4922-a75a-442f73b1caec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:36.283414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef88b0b-260c-4922-a75a-442f73b1caec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:36.283414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef88b0b-260c-4922-a75a-442f73b1caec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:36.299750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:36.300563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:36.300735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:36.301398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:36.302196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:36.303940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:36.326483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 817/1639] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:36.327838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:36.354364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cef88b0b-260c-4922-a75a-442f73b1caec None None] MFA Rules not processed for user `f7fbe9450cc445abb13d2e7ca654a099`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:36.359189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cef88b0b-260c-4922-a75a-442f73b1caec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:36.401287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:36.434576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:36.435375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e6afc-b3e8-4485-8627-9db0c174b9ed admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:36.454504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542a5fb-c231-47a5-b99d-ae37ce293822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:36.455439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542a5fb-c231-47a5-b99d-ae37ce293822 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:36.455725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542a5fb-c231-47a5-b99d-ae37ce293822 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:36.455995 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542a5fb-c231-47a5-b99d-ae37ce293822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:36.483654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542a5fb-c231-47a5-b99d-ae37ce293822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:36.485138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542a5fb-c231-47a5-b99d-ae37ce293822 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:36.497180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 823/1640] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:36.501287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef88b0b-260c-4922-a75a-442f73b1caec None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:36.504925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 818/1641] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:36.512394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef88b0b-260c-4922-a75a-442f73b1caec None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:36.518208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:36.521663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:36.521975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:36.526388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] PATH_INFO: `/v3/projects/ce58f0db51124e5a805f8e273d2dde40/users/72a5a6ac38e44c558345315a4ec50a1c/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:36.530670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce58f0db51124e5a805f8e273d2dde40, user_id=72a5a6ac38e44c558345315a4ec50a1c, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:36.531707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:36.559262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 824/1642] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:36 2022] PUT /identity/v3/projects/ce58f0db51124e5a805f8e273d2dde40/users/72a5a6ac38e44c558345315a4ec50a1c/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:36.574477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e25eb9-94ab-46cd-8c09-6cd587095b23 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:36.574888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e25eb9-94ab-46cd-8c09-6cd587095b23 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:36.575237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e25eb9-94ab-46cd-8c09-6cd587095b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:36.667655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:36.668238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-26e25eb9-94ab-46cd-8c09-6cd587095b23 None None] MFA Rules not processed for user `72a5a6ac38e44c558345315a4ec50a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:36.673984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-26e25eb9-94ab-46cd-8c09-6cd587095b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:36.765690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a236fa7-5689-4575-a6ae-a86405abb52d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:36.769047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:36.769860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:36.770213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:36.770596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:36.790764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 819/1643] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:36.901535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:36.903526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:36.927211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 825/1644] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:13:37.238037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e25eb9-94ab-46cd-8c09-6cd587095b23 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:37.238037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e25eb9-94ab-46cd-8c09-6cd587095b23 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:37.238037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:37.238037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:37.240920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:37.240920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:37.240920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:37.240920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:37.240920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 820/1645] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:37.432158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:37.444100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc35548-deb1-4cea-81f5-8b8777432018 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:37.454010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0d806-be18-4a56-98b7-204c1cd67b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:37.454686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0d806-be18-4a56-98b7-204c1cd67b6f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:37.455196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0d806-be18-4a56-98b7-204c1cd67b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:37.455741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0d806-be18-4a56-98b7-204c1cd67b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:37.575564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0d806-be18-4a56-98b7-204c1cd67b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:37.578905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0d806-be18-4a56-98b7-204c1cd67b6f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:37.592259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 826/1646] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:37.625188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:37.640990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed911-e99a-40b1-9085-d3e7aad0549f service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:37.652564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f850b-ba31-4c10-89c0-0d524ee46505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:37.653227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f850b-ba31-4c10-89c0-0d524ee46505 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:37.653514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f850b-ba31-4c10-89c0-0d524ee46505 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:37.657770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f850b-ba31-4c10-89c0-0d524ee46505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:37.777385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd824-360d-466f-8673-d57b8f451c61 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:37.777819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd824-360d-466f-8673-d57b8f451c61 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:37.778210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd824-360d-466f-8673-d57b8f451c61 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:37.780402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 827/1647] 10.222.0.39 () {54 vars in 962 bytes} [Sun Sep 4 16:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:13:37.795273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb9d7c-8696-40c6-87a9-10efa66de721 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:37.795882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb9d7c-8696-40c6-87a9-10efa66de721 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:37.796241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb9d7c-8696-40c6-87a9-10efa66de721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:37.811099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f850b-ba31-4c10-89c0-0d524ee46505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:37.815848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f850b-ba31-4c10-89c0-0d524ee46505 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:37.822712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:37.835979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 821/1648] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:37.851376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aaeb9d7c-8696-40c6-87a9-10efa66de721 None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:37.851376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aaeb9d7c-8696-40c6-87a9-10efa66de721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:37.994313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 828/1649] 10.222.0.39 () {58 vars in 1059 bytes} [Sun Sep 4 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:38.002359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f82dd-dc6f-48ba-900c-72c50d33a9a5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:38.002795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f82dd-dc6f-48ba-900c-72c50d33a9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:38.006219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f82dd-dc6f-48ba-900c-72c50d33a9a5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:38.006219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 822/1650] 10.222.0.39 () {54 vars in 912 bytes} [Sun Sep 4 16:13:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:13:38.015647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbdaf-ef37-4786-a74a-d43ab460e19b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:38.016606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbdaf-ef37-4786-a74a-d43ab460e19b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:38.016908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbdaf-ef37-4786-a74a-d43ab460e19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:38.052304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f90bbdaf-ef37-4786-a74a-d43ab460e19b None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:38.069737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f90bbdaf-ef37-4786-a74a-d43ab460e19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:38.100063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 829/1651] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:38.408595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f82dd-dc6f-48ba-900c-72c50d33a9a5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:38.429393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f82dd-dc6f-48ba-900c-72c50d33a9a5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:38.438225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:38.439977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:38.441374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:38.441856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:38.479504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:38.481141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:38.524150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 823/1652] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:39.724239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bbdaf-ef37-4786-a74a-d43ab460e19b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:39.923336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bbdaf-ef37-4786-a74a-d43ab460e19b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:39.942104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:39.943123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:39.943617 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:39.944081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] PATH_INFO: `/v3/users/35a0b70ddb65478194fa600e92018265` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:39.947409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a0b70ddb65478194fa600e92018265)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:39.948714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:13:39.948714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Sep 04 16:13:39.948714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: Sep 04 16:13:39.950379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:40.357890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a0b70ddb65478194fa600e92018265'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.359464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a0b70ddb65478194fa600e92018265'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.421002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a0b70ddb65478194fa600e92018265'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.491463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:40.497051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a0b70ddb65478194fa600e92018265'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.498002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c0ac21-2855-447d-afe1-b080743c0079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a0b70ddb65478194fa600e92018265'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.511120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90050-3b1b-41df-b514-c5acc60bb000 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:40.533714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:40.534204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 830/1653] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:13:39 2022] DELETE /identity/v3/users/35a0b70ddb65478194fa600e92018265 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:40.535795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:40.541340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:40.541340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] PATH_INFO: `/v3/users/dd48767bd3fa451fbeb5bbf40084d6b0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:40.544180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd48767bd3fa451fbeb5bbf40084d6b0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:40.544989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:40.545378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:40.545805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:40.549350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:40.585631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:40.586732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:40.619053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:40.754778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 831/1654] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:40.777974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:40.783468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:40.842048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:40.875124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd48767bd3fa451fbeb5bbf40084d6b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.875124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd48767bd3fa451fbeb5bbf40084d6b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.899327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcffd7-3b7f-40b0-b83a-2870ae3046bc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:40.915671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b4d15-6630-48dd-b7c7-5b75f05801d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:40.916432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b4d15-6630-48dd-b7c7-5b75f05801d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:40.916883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b4d15-6630-48dd-b7c7-5b75f05801d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:40.917260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b4d15-6630-48dd-b7c7-5b75f05801d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:40.919569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-328b4d15-6630-48dd-b7c7-5b75f05801d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:40.968119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b4d15-6630-48dd-b7c7-5b75f05801d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:40.972517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b4d15-6630-48dd-b7c7-5b75f05801d7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:40.979716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd48767bd3fa451fbeb5bbf40084d6b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:40.980365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd48767bd3fa451fbeb5bbf40084d6b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:41.012769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd48767bd3fa451fbeb5bbf40084d6b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:41.033851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 832/1655] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:41.042989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 824/1656] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:13:40 2022] DELETE /identity/v3/users/dd48767bd3fa451fbeb5bbf40084d6b0 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:41.055538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:41.056081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:41.056595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:41.094873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:41.096491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:41.242348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 833/1657] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:41.289363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:41.291462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:41.475029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d444fd-6c95-43b7-b0b7-033fb090f07b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:41.489300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:41.491214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:41.491511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:41.492202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:41.521542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:41.527156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:41.542953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 825/1658] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:43.682897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:43.686119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:43.713779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b91ac-a4bb-499a-9950-58746cda26a9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:43.748605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:43.751354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:43.758646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:43.762441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] PATH_INFO: `/v3/projects/2f68c2eedf0b4c52ba46d302f28c92ee` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:43.767341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f68c2eedf0b4c52ba46d302f28c92ee)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:43.773644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:44.012512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f68c2eedf0b4c52ba46d302f28c92ee', 'request_id': 'req-865e3507-9876-4add-8be5-8c965bda58ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.014315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:44.038485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f68c2eedf0b4c52ba46d302f28c92ee', 'request_id': 'req-865e3507-9876-4add-8be5-8c965bda58ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.062340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f68c2eedf0b4c52ba46d302f28c92ee', 'request_id': 'req-865e3507-9876-4add-8be5-8c965bda58ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.169392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] The token cache is being invalidate because project 2f68c2eedf0b4c52ba46d302f28c92ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:13:44.169392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.169392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.169905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 834/1659] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:13:43 2022] DELETE /identity/v3/projects/2f68c2eedf0b4c52ba46d302f28c92ee => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:44.173803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:44.190843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5598-b844-4c4d-8cd4-4b095cf31213 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:44.343381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:44.344123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:44.347968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:44.349161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] PATH_INFO: `/v3/users/72a5a6ac38e44c558345315a4ec50a1c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:44.354947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a5a6ac38e44c558345315a4ec50a1c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:44.357387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:44.638750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a5a6ac38e44c558345315a4ec50a1c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.643697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a5a6ac38e44c558345315a4ec50a1c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.739681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a5a6ac38e44c558345315a4ec50a1c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.740336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a5a6ac38e44c558345315a4ec50a1c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.747234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:44.785154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-de6e081d-a5dd-48b0-a109-5dc76c8a6a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a5a6ac38e44c558345315a4ec50a1c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:44.828536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 826/1660] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:13:44 2022] DELETE /identity/v3/users/72a5a6ac38e44c558345315a4ec50a1c => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:44.866180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:44.867774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:44.890532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e3507-9876-4add-8be5-8c965bda58ea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:45.004873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:45.005886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:45.006313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:45.006731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] PATH_INFO: `/v3/projects/f7786c29c31642a4a02db494e8a57e59` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:45.008724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7786c29c31642a4a02db494e8a57e59)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:45.012932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:45.193143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7786c29c31642a4a02db494e8a57e59', 'request_id': 'req-22c4e2c9-e25e-4496-9fe8-76747727866f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:45.222042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7786c29c31642a4a02db494e8a57e59', 'request_id': 'req-22c4e2c9-e25e-4496-9fe8-76747727866f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:45.232501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7786c29c31642a4a02db494e8a57e59', 'request_id': 'req-22c4e2c9-e25e-4496-9fe8-76747727866f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:45.340980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] The token cache is being invalidate because project f7786c29c31642a4a02db494e8a57e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:13:45.341286 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:45.343129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-22c4e2c9-e25e-4496-9fe8-76747727866f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:45.345414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 835/1661] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:13:44 2022] DELETE /identity/v3/projects/f7786c29c31642a4a02db494e8a57e59 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:45.350179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67716278-53b9-4d81-b15c-c863df23acda None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:45.350707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67716278-53b9-4d81-b15c-c863df23acda None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:45.350865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67716278-53b9-4d81-b15c-c863df23acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:45.402866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-67716278-53b9-4d81-b15c-c863df23acda None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:45.404318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-67716278-53b9-4d81-b15c-c863df23acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:45.646156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 827/1662] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:45.653557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f001f-0ffa-469c-ae38-03cb984529ee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:45.653902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f001f-0ffa-469c-ae38-03cb984529ee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:45.654214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f001f-0ffa-469c-ae38-03cb984529ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:45.693434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d99f001f-0ffa-469c-ae38-03cb984529ee None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:45.695083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d99f001f-0ffa-469c-ae38-03cb984529ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:45.705071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 836/1663] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:45.709142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67716278-53b9-4d81-b15c-c863df23acda None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:45.736646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67716278-53b9-4d81-b15c-c863df23acda None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:45.752861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:45.754307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:45.756183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:45.757145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:45.759054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:45.761253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:45.766230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:45.806391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 828/1664] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:45.810540 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f001f-0ffa-469c-ae38-03cb984529ee None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:45.826092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f001f-0ffa-469c-ae38-03cb984529ee None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:45.832582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:45.833508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:45.834453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:45.835005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:45.835871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:45.838333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:45.960214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 837/1665] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:45 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:13:45.964411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:45.984044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4644df-8c24-401e-bf10-f1d0ecff2bc2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:46.139050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:46.139695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:46.139977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:46.142824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:46.143573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:46.150393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:46.246442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 829/1666] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:45 2022] POST /identity/v3/users => generated 303 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:46.251209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:46.270578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a854731-141b-42e8-bed5-7b9e98a044c9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:46.285487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:46.286163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:46.286861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:46.287148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:46.287811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:46.290719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:46.317933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 838/1667] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:46.321332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:46.338773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dba300-3d5a-452d-a618-1ce234725627 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:46.353116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937582db-b9a4-4b2b-a459-3fd4c8f524b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:46.353744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937582db-b9a4-4b2b-a459-3fd4c8f524b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:46.354098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937582db-b9a4-4b2b-a459-3fd4c8f524b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:46.354455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937582db-b9a4-4b2b-a459-3fd4c8f524b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:46.355191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937582db-b9a4-4b2b-a459-3fd4c8f524b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:46.359203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937582db-b9a4-4b2b-a459-3fd4c8f524b7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:46.386773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 830/1668] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:46.387484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:46.403299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae8bda-c86b-41b8-bd82-0de7cbcabf54 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:46.419511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3a451-9797-419e-89de-eab404a48999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:46.420202 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3a451-9797-419e-89de-eab404a48999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:46.420485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3a451-9797-419e-89de-eab404a48999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:46.420764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3a451-9797-419e-89de-eab404a48999 admin admin] PATH_INFO: `/v3/projects/444b2403d08a4782a56a5543d6c50029/users/bf729c8e202544d4b4000e8adc5a0da3/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:46.428566 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:46.431742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:46.431742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:46.454452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3a451-9797-419e-89de-eab404a48999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444b2403d08a4782a56a5543d6c50029, user_id=bf729c8e202544d4b4000e8adc5a0da3, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:46.460213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3a451-9797-419e-89de-eab404a48999 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:46.503832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] MFA Rules not processed for user `1623e93ad6844cacae7cc1a71b339f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:46.508531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:46.583192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 839/1669] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:46 2022] PUT /identity/v3/projects/444b2403d08a4782a56a5543d6c50029/users/bf729c8e202544d4b4000e8adc5a0da3/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:13:46.588445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:46.588445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:46.589079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:46.589529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:46.613424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:46.663233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:46.664627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:46.687335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 831/1670] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:46.699804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:46.701300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:46.824805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 840/1671] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:46.825728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:46.829712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:46.849606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cbfdd-895f-4c54-8fa2-4ae15601affb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:46.862268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:46.863159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:46.863701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:46.864113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] PATH_INFO: `/v3/projects/ce58f0db51124e5a805f8e273d2dde40` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:46.866101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce58f0db51124e5a805f8e273d2dde40)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:46.887217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:46.893318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc4fdf-ce82-44d9-ae33-9f159bf4668a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:47.005415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:47.006357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:47.006711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:47.007095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:47.078043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce58f0db51124e5a805f8e273d2dde40', 'request_id': 'req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:47.093126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce58f0db51124e5a805f8e273d2dde40', 'request_id': 'req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:47.107863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce58f0db51124e5a805f8e273d2dde40', 'request_id': 'req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:47.167471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:47.169455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:47.182753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 832/1672] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:47.183473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:47.195086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:47.219829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] The token cache is being invalidate because project ce58f0db51124e5a805f8e273d2dde40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:13:47.220321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:47.221273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b6974da0-03d3-4d1a-ad28-e0c6e9f649ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:47.223670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 841/1673] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:13:46 2022] DELETE /identity/v3/projects/ce58f0db51124e5a805f8e273d2dde40 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:47.227730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d760e3-8d54-411d-b648-42fec650441d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:47.228151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d760e3-8d54-411d-b648-42fec650441d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:47.231561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d760e3-8d54-411d-b648-42fec650441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:47.311142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-24d760e3-8d54-411d-b648-42fec650441d None None] MFA Rules not processed for user `f1af03ad31a34f319763a6b5451ae26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:47.315663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-24d760e3-8d54-411d-b648-42fec650441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:47.333554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d3c08-65ab-41ba-a857-cfb1d4b51450 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:47.580858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d91115-5854-4ead-b96c-865ea95f5100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:47.584141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d91115-5854-4ead-b96c-865ea95f5100 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:47.586485 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d91115-5854-4ead-b96c-865ea95f5100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:47.586818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d91115-5854-4ead-b96c-865ea95f5100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:47.596144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2d91115-5854-4ead-b96c-865ea95f5100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:47.623819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 842/1674] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:47.625337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a80089-0bd7-4457-bddd-2373c1bf1adb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:47.625720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a80089-0bd7-4457-bddd-2373c1bf1adb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:47.626005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a80089-0bd7-4457-bddd-2373c1bf1adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:47.722535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d6a80089-0bd7-4457-bddd-2373c1bf1adb None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:47.723797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d6a80089-0bd7-4457-bddd-2373c1bf1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:47.873765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d91115-5854-4ead-b96c-865ea95f5100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:47.875226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d91115-5854-4ead-b96c-865ea95f5100 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:47.923312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 833/1675] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:47.927219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:47.927514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:47.927796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:47.964055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 843/1676] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:47.984818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f1bf-3969-4cb3-b70f-b5ef015eef74 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:47.985147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f1bf-3969-4cb3-b70f-b5ef015eef74 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:47.987262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f1bf-3969-4cb3-b70f-b5ef015eef74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:47.996019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] MFA Rules not processed for user `f1af03ad31a34f319763a6b5451ae26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:47.997462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:48.015123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 834/1677] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:48.022211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:48.025250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:48.058077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-17c3f1bf-3969-4cb3-b70f-b5ef015eef74 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:48.060444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-17c3f1bf-3969-4cb3-b70f-b5ef015eef74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:48.076083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 844/1678] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:48.080675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3f1bf-3969-4cb3-b70f-b5ef015eef74 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:48.110251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3f1bf-3969-4cb3-b70f-b5ef015eef74 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:48.150021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:48.152015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:48.152852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:48.154861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:48.155901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:48.157017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:48.183704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:48.218599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 845/1679] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:48.218599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:48.238535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e40a0-4bcd-4090-9fdc-72dd5134f685 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:48.255090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d217ea-b205-49af-959a-b45de93807c1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:48.262662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:48.263969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:48.264632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:48.267064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:48.268108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:48.269326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:48.270182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:48.270573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:48.270985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:48.273077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:48.284778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:48.286665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:48.325120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 835/1680] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:13:48.346997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:48.423919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 846/1681] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:48 2022] POST /identity/v3/projects => generated 363 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:48.425945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:48.434360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:48.446534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2cef6-2861-4e7e-a4ba-57901f361501 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:48.671524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:48.672168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:48.672587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:48.673487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:48.674260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:48.683482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:48.748922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:48.839521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:48.844842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 836/1682] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:48 2022] POST /identity/v3/users => generated 282 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:48.855537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a62e6-3154-4c8b-94e6-6abcff4904af admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:48.874065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:48.874997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:48.875481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:48.877635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:48.877635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:48.878942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:48.908751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 847/1683] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:48.911108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:48.925683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0927006-0eee-457d-bae2-293dc34805ba admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:48.936853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1c60a-024f-47a7-b9c0-818fc6b895f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:48.937494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1c60a-024f-47a7-b9c0-818fc6b895f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:48.937812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1c60a-024f-47a7-b9c0-818fc6b895f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:48.938753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1c60a-024f-47a7-b9c0-818fc6b895f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:48.939423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1c60a-024f-47a7-b9c0-818fc6b895f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:48.941082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1c60a-024f-47a7-b9c0-818fc6b895f9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:48.966006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 837/1684] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:48.970358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:49.003227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95bde-6ed2-4687-a99e-feecbaf7e2cd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:49.015154 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:49.016620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:49.018283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:49.018641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] PATH_INFO: `/v3/projects/ed62d2d3650047aba9ca095b6a3953fe/users/872338265dbc453ebccf6fd93f9b21b6/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:49.060266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed62d2d3650047aba9ca095b6a3953fe, user_id=872338265dbc453ebccf6fd93f9b21b6, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:49.067582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:49.164956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 848/1685] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:48 2022] PUT /identity/v3/projects/ed62d2d3650047aba9ca095b6a3953fe/users/872338265dbc453ebccf6fd93f9b21b6/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:49.179231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada564b-8c26-4590-8f58-4708e0444b81 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:49.179623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada564b-8c26-4590-8f58-4708e0444b81 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:49.180005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada564b-8c26-4590-8f58-4708e0444b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:49.268700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8ada564b-8c26-4590-8f58-4708e0444b81 None None] MFA Rules not processed for user `872338265dbc453ebccf6fd93f9b21b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:49.270171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ada564b-8c26-4590-8f58-4708e0444b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:49.392027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 838/1686] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:49.393477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:49.505943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da266752-7741-400c-8f24-fef4b032b366 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:49.506241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da266752-7741-400c-8f24-fef4b032b366 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:49.506543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da266752-7741-400c-8f24-fef4b032b366 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:49.508528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 839/1687] 10.222.0.39 () {54 vars in 912 bytes} [Sun Sep 4 16:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:13:49.517977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:49.518533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:49.518961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:49.547566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4303234-dc29-42f3-b3a8-2de5880f5900 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:49.574486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:49.575415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:49.575708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:49.575991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:49.576676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:49.583133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:49.674404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] MFA Rules not processed for user `93ef027007184b8180325d6fb26324a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:49.679845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:49.732362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 849/1688] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:49 2022] POST /identity/v3/projects => generated 363 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:49.737409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:49.822931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:49.846069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 840/1689] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:49.862600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:49.864095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:49.890580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b557d27-6fa4-4859-8069-31297c75bb48 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:49.938548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812d809-054e-4453-a162-2c976c885eca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:49.952151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:49.952986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:49.953419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:49.954517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:49.956019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:49.958557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:50.039476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:50.040116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:50.040399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:50.041971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:50.059889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:50.067041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:50.084936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 841/1690] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:13:50.088694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 850/1691] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:49 2022] POST /identity/v3/users => generated 288 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:50.092616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:50.105894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a449373-7f85-43b5-a035-588f734e9063 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:50.111565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:50.112237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:50.113086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:50.113086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:50.113427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:50.115072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:50.117318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:50.133479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 842/1692] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:50.137323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:50.151544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493efcc-2653-42fb-95df-a12c7953a0fc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:50.172044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:50.174370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:50.176428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:50.176732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] PATH_INFO: `/v3/projects/e774896864974d3da13f42fcef8004d7/users/e5bc57508d094bef86b64637e0160de9/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:50.223074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e774896864974d3da13f42fcef8004d7, user_id=e5bc57508d094bef86b64637e0160de9, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:50.231551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:50.360387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 851/1693] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:50 2022] PUT /identity/v3/projects/e774896864974d3da13f42fcef8004d7/users/e5bc57508d094bef86b64637e0160de9/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:50.360611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:50.378502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cc593-1784-44e2-a4e6-c75712cd46e6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:50.536793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4af16b-7745-42c0-a039-0082f92478f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:50.537427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4af16b-7745-42c0-a039-0082f92478f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:50.537792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4af16b-7745-42c0-a039-0082f92478f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:50.538067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4af16b-7745-42c0-a039-0082f92478f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:50.539823 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4af16b-7745-42c0-a039-0082f92478f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:50.541863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4af16b-7745-42c0-a039-0082f92478f9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:50.554773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 843/1694] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:50.565809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:50.582599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb91c-9467-45ef-a3c9-143eb9f83f88 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:50.594322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:50.595284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:50.595572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:50.596904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] PATH_INFO: `/v3/projects/e774896864974d3da13f42fcef8004d7/users/e5bc57508d094bef86b64637e0160de9/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:50.599252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e774896864974d3da13f42fcef8004d7, user_id=e5bc57508d094bef86b64637e0160de9, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:50.602214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:50.631435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 852/1695] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:50 2022] PUT /identity/v3/projects/e774896864974d3da13f42fcef8004d7/users/e5bc57508d094bef86b64637e0160de9/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:13:50.643581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:50.643945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:50.644293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:50.728051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] MFA Rules not processed for user `e5bc57508d094bef86b64637e0160de9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:50.731103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:50.936057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 844/1696] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:51.014029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:51.021696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:51.147837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:51.150869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:51.171220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:51.223348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dad4d-5bcd-40a2-9cf1-19117612f359 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:51.234425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c218460-5d63-414b-a76c-598a44517df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:51.237157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c218460-5d63-414b-a76c-598a44517df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:51.238542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c218460-5d63-414b-a76c-598a44517df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:51.238831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c218460-5d63-414b-a76c-598a44517df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:51.273789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c218460-5d63-414b-a76c-598a44517df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:51.282553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c218460-5d63-414b-a76c-598a44517df5 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:51.307631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 853/1697] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:51.393097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b49c6b-9eac-46d5-953e-7de0aca85ea6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:51.408889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:51.409629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:51.409921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:51.410204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:51.424162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:51.462712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:51.464437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:51.485980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 845/1698] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:52.294830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:52.295379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:52.295379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:52.401704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] MFA Rules not processed for user `bf729c8e202544d4b4000e8adc5a0da3`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:52.404426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:52.520232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:52.522325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:52.570374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 854/1699] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:52.578276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:52.580742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:52.612282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:52.637139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db334-3fd0-4118-b0be-e535ac8c16fe service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:52.652318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:52.652505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:52.653011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:52.653363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] PATH_INFO: `/v3/users/5f8aa81e13da42a8bb35f8a983033a91` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:52.654779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8aa81e13da42a8bb35f8a983033a91)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:52.656956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:52.767367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2082f5-e4df-408b-82f1-1a62d56a5e53 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:52.774103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc0aa3-dfa3-4d1e-a5cc-54c1eb96bf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:52.774899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc0aa3-dfa3-4d1e-a5cc-54c1eb96bf29 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:52.775190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc0aa3-dfa3-4d1e-a5cc-54c1eb96bf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:52.775655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc0aa3-dfa3-4d1e-a5cc-54c1eb96bf29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:52.776315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc0aa3-dfa3-4d1e-a5cc-54c1eb96bf29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:52.778527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc0aa3-dfa3-4d1e-a5cc-54c1eb96bf29 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:52.890684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8aa81e13da42a8bb35f8a983033a91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:52.902421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 855/1700] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:52 2022] POST /identity/v3/projects => generated 405 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:13:52.906924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:52.907226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:52.907588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:52.910281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8aa81e13da42a8bb35f8a983033a91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:52.962635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] MFA Rules not processed for user `a9d55c52717546b09d93d956a29f49a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:52.964522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.006200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8aa81e13da42a8bb35f8a983033a91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.006867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8aa81e13da42a8bb35f8a983033a91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.031483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8aa81e13da42a8bb35f8a983033a91'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.073835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 846/1701] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:13:52 2022] DELETE /identity/v3/users/5f8aa81e13da42a8bb35f8a983033a91 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:53.074844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:53.079875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.139310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:53.153243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 856/1702] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:53.154215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:53.158852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.249215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1558b65-d9c2-4704-958d-f79a69d9b2d4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:53.259245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:53.259901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:53.260181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:53.261367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] PATH_INFO: `/v3/users/872338265dbc453ebccf6fd93f9b21b6` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:53.262061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=872338265dbc453ebccf6fd93f9b21b6)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:53.263990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:53.327501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfa029-e684-4547-bef6-8048871333a4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:53.338789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf459a48-f1da-4c1b-ba63-60922980c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:53.339428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf459a48-f1da-4c1b-ba63-60922980c2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:53.339720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf459a48-f1da-4c1b-ba63-60922980c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:53.339989 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf459a48-f1da-4c1b-ba63-60922980c2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:53.340693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf459a48-f1da-4c1b-ba63-60922980c2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:53.345777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf459a48-f1da-4c1b-ba63-60922980c2b1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:53.467019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 857/1703] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:53 2022] POST /identity/v3/users => generated 309 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:53.471389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af446d55-09ef-43a3-a262-e4acfcf2bb4d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:53.471953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af446d55-09ef-43a3-a262-e4acfcf2bb4d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:53.472422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af446d55-09ef-43a3-a262-e4acfcf2bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:53.527620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-af446d55-09ef-43a3-a262-e4acfcf2bb4d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:53.530498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af446d55-09ef-43a3-a262-e4acfcf2bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.564811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 858/1704] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:53.568630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:53.569127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:53.569751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:53.600219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872338265dbc453ebccf6fd93f9b21b6'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.603347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872338265dbc453ebccf6fd93f9b21b6'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.627049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] MFA Rules not processed for user `a9d55c52717546b09d93d956a29f49a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:53.628281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.639240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:53.738354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872338265dbc453ebccf6fd93f9b21b6'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.741543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872338265dbc453ebccf6fd93f9b21b6'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.789718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872338265dbc453ebccf6fd93f9b21b6'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:53.819754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 859/1705] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:53.822231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:53.825871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.832778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 847/1706] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:13:53 2022] DELETE /identity/v3/users/872338265dbc453ebccf6fd93f9b21b6 => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:53.834424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:53.843195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.877783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0dc4f-2f26-49eb-9635-56fc3f1b1f0d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:53.967699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60ed88-287f-4844-b8a9-558f23c571b9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:53.973812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323810a-ffee-47a1-a96f-462b76d6c7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:53.975056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323810a-ffee-47a1-a96f-462b76d6c7df service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:53.975433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323810a-ffee-47a1-a96f-462b76d6c7df service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:53.975786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323810a-ffee-47a1-a96f-462b76d6c7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:53.977788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a323810a-ffee-47a1-a96f-462b76d6c7df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:53.990502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:53.991167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:53.991461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:53.991725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:53.993696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:53.994103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:53.994747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323810a-ffee-47a1-a96f-462b76d6c7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:53.996747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323810a-ffee-47a1-a96f-462b76d6c7df service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:54.009883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 848/1707] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:54.011880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f499f-ffcd-419c-abd2-7fe3ed4444c2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:54.018223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 860/1708] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:54.018223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:54.029819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f499f-ffcd-419c-abd2-7fe3ed4444c2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:54.029819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f499f-ffcd-419c-abd2-7fe3ed4444c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:54.036041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:54.113903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0f4f499f-ffcd-419c-abd2-7fe3ed4444c2 None None] MFA Rules not processed for user `6e5c22b631df4600b9f920120b67fc67`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:54.115881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0f4f499f-ffcd-419c-abd2-7fe3ed4444c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:54.289998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:54.322747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda0e1-3823-4ec4-b6a1-19659509d765 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:54.331314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 849/1709] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:54.336048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:54.336740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:54.337063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:54.337287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:54.338685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73eb446-c613-4edd-8cf0-ef48685a7578 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:54.338978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73eb446-c613-4edd-8cf0-ef48685a7578 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:54.339264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73eb446-c613-4edd-8cf0-ef48685a7578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:54.342768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:54.393212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b73eb446-c613-4edd-8cf0-ef48685a7578 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:54.394430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b73eb446-c613-4edd-8cf0-ef48685a7578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:54.410461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 850/1710] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:54.414873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73eb446-c613-4edd-8cf0-ef48685a7578 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:54.426356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73eb446-c613-4edd-8cf0-ef48685a7578 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:54.431753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:54.432484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:54.432772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:54.433114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] PATH_INFO: `/v3/projects/689b750af589492eb8ffef7969a92282/users/a27f65d219f84286987f1266c88de1ea/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:54.452490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689b750af589492eb8ffef7969a92282, user_id=a27f65d219f84286987f1266c88de1ea, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:54.456272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:54.491336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:54.515220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:54.517201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:54.538903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 861/1711] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:54.540544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:54.542367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:54.570954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac3206-af2f-4602-abf1-3e489708da66 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:54.577080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 851/1712] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:54 2022] PUT /identity/v3/projects/689b750af589492eb8ffef7969a92282/users/a27f65d219f84286987f1266c88de1ea/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:54.577080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:54.762848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c657d-5f07-4102-9928-dd96a0ec40f0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:54.772858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:54.773496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:54.773829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:54.774196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:54.889217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:54.890438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:54.891679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:54.896669 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:54.957908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:54.969110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:54.997070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 852/1713] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:54.998210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:55.009282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:55.012761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:55.020617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718a62b-f427-4c21-aa87-67c98e464938 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:55.030064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 862/1714] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:13:55.030605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:55.031014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:55.033041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:55.033290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:55.033722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:55.034801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:55.036730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:55.045133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:55.066954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04825398-b750-4d90-a2ec-3efeb6a81b47 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:55.073606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 853/1715] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:13:55.076269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:55.079093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:55.086260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:55.087032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:55.088048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] PATH_INFO: `/v3/projects/183fb107080444bf99d0369dde5e5466` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:55.091267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f117a1-6bfd-4418-b770-ec2adbd87c41 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:55.093447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=183fb107080444bf99d0369dde5e5466)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:55.098705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc47b8-7172-4cec-83aa-5b5a9f8378fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:55.099392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc47b8-7172-4cec-83aa-5b5a9f8378fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:55.099664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:55.099770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc47b8-7172-4cec-83aa-5b5a9f8378fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:55.100023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc47b8-7172-4cec-83aa-5b5a9f8378fe admin admin] PATH_INFO: `/v3/projects/689b750af589492eb8ffef7969a92282/users/a27f65d219f84286987f1266c88de1ea/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:55.102428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc47b8-7172-4cec-83aa-5b5a9f8378fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=689b750af589492eb8ffef7969a92282, user_id=a27f65d219f84286987f1266c88de1ea, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:55.105341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc47b8-7172-4cec-83aa-5b5a9f8378fe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:55.122961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 854/1716] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:13:55 2022] PUT /identity/v3/projects/689b750af589492eb8ffef7969a92282/users/a27f65d219f84286987f1266c88de1ea/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:55.320867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183fb107080444bf99d0369dde5e5466', 'request_id': 'req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:55.346337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183fb107080444bf99d0369dde5e5466', 'request_id': 'req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:55.359111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183fb107080444bf99d0369dde5e5466', 'request_id': 'req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:55.395445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aaad6-01b9-4306-9efa-d2b6e26d5b30 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:55.395964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aaad6-01b9-4306-9efa-d2b6e26d5b30 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:55.396725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aaad6-01b9-4306-9efa-d2b6e26d5b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:55.437400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ca5aaad6-01b9-4306-9efa-d2b6e26d5b30 None None] MFA Rules not processed for user `1623e93ad6844cacae7cc1a71b339f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:55.440902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca5aaad6-01b9-4306-9efa-d2b6e26d5b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:55.477185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] The token cache is being invalidate because project 183fb107080444bf99d0369dde5e5466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:13:55.477785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:55.478605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:55.480155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 863/1717] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:13:55 2022] DELETE /identity/v3/projects/183fb107080444bf99d0369dde5e5466 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:55.485137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:55.488598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:55.513019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:55.626327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 855/1718] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:55.656530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa5170-e470-47e6-a9c8-0999c6c8fdd4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:55.662275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:55.663059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:55.663802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:55.663802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] PATH_INFO: `/v3/users/184ae94126674ddeae0aa365f92831ce` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:55.664983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=184ae94126674ddeae0aa365f92831ce)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:55.667401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:55.930959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184ae94126674ddeae0aa365f92831ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:55.933197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184ae94126674ddeae0aa365f92831ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:55.967097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184ae94126674ddeae0aa365f92831ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:56.052682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184ae94126674ddeae0aa365f92831ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:56.054064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184ae94126674ddeae0aa365f92831ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:56.076809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:56.086980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 864/1719] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:13:55 2022] DELETE /identity/v3/users/184ae94126674ddeae0aa365f92831ce => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:56.183818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192b0c-7261-4415-b6ac-64fb6746ebf0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:56.184723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192b0c-7261-4415-b6ac-64fb6746ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:56.185225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23192b0c-7261-4415-b6ac-64fb6746ebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:56.236395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-23192b0c-7261-4415-b6ac-64fb6746ebf0 None None] MFA Rules not processed for user `a9d55c52717546b09d93d956a29f49a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:56.238356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-23192b0c-7261-4415-b6ac-64fb6746ebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:56.449552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 856/1720] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:57.009443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:57.012450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:57.143617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac865a5-7c4d-43ad-bd2b-c9275309d5b0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:57.144184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac865a5-7c4d-43ad-bd2b-c9275309d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:57.145197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac865a5-7c4d-43ad-bd2b-c9275309d5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:57.220459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb5766-e479-4b64-a829-891d308987ac admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:57.245083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:57.246033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:57.246315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:57.246743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] PATH_INFO: `/v3/projects/ed62d2d3650047aba9ca095b6a3953fe` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:57.250223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed62d2d3650047aba9ca095b6a3953fe)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:57.254790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:57.295986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fac865a5-7c4d-43ad-bd2b-c9275309d5b0 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:57.298419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fac865a5-7c4d-43ad-bd2b-c9275309d5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:57.475344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 857/1721] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:13:57.525670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:57.556394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed62d2d3650047aba9ca095b6a3953fe', 'request_id': 'req-2aea5e1c-0d72-4d9b-8364-63200b44714a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:57.570289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed62d2d3650047aba9ca095b6a3953fe', 'request_id': 'req-2aea5e1c-0d72-4d9b-8364-63200b44714a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:57.586538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed62d2d3650047aba9ca095b6a3953fe', 'request_id': 'req-2aea5e1c-0d72-4d9b-8364-63200b44714a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:57.632927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd56f94-79c2-46c5-93d8-d2cabf7e8447 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:57.633319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd56f94-79c2-46c5-93d8-d2cabf7e8447 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:57.633492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd56f94-79c2-46c5-93d8-d2cabf7e8447 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:57.635034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 858/1722] 10.222.0.39 () {54 vars in 912 bytes} [Sun Sep 4 16:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 16:13:57.643300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:57.643619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:57.643958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:57.679533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] MFA Rules not processed for user `93ef027007184b8180325d6fb26324a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:57.680779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:57.708633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] The token cache is being invalidate because project ed62d2d3650047aba9ca095b6a3953fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:13:57.708943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:57.711158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:57.715289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 865/1723] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:13:57 2022] DELETE /identity/v3/projects/ed62d2d3650047aba9ca095b6a3953fe => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:57.719613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:57.725789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:57.766088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:57.789581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 859/1724] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:57.851372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea5e1c-0d72-4d9b-8364-63200b44714a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:57.866661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:57.866661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:57.866661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:57.866995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] PATH_INFO: `/v3/users/e5bc57508d094bef86b64637e0160de9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:57.869310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5bc57508d094bef86b64637e0160de9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:57.872153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:57.906680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:57.908762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:57.944950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047bdb-e8f2-4ede-9639-932ed0a74c60 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:57.975992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:57.975992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:57.975992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:57.976198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:58.005010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:58.010352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:58.030398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 860/1725] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:13:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:13:58.031286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:58.038234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:58.294057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5bc57508d094bef86b64637e0160de9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:58.298783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5bc57508d094bef86b64637e0160de9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:58.372263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5bc57508d094bef86b64637e0160de9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:58.404158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf468e-13b1-42ef-b464-689f26fafee9 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:58.418431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7504e-d3c3-4c14-b841-256946826b46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:58.420575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7504e-d3c3-4c14-b841-256946826b46 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:58.424842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7504e-d3c3-4c14-b841-256946826b46 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:58.424842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7504e-d3c3-4c14-b841-256946826b46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:58.429073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b2b7504e-d3c3-4c14-b841-256946826b46 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:58.460920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5bc57508d094bef86b64637e0160de9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:58.461850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5bc57508d094bef86b64637e0160de9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:58.486193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 866/1726] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:13:57 2022] DELETE /identity/v3/users/e5bc57508d094bef86b64637e0160de9 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:13:58.487367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:58.488939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:58.624741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7504e-d3c3-4c14-b841-256946826b46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:58.627402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7504e-d3c3-4c14-b841-256946826b46 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:58.642352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886744c8-0025-4794-90cd-b61730f28f78 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:58.653520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 861/1727] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:13:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:13:58.686077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:58.687294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:58.692337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:58.693626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] PATH_INFO: `/v3/projects/89173a8fed6b4e6381a006abae66baa7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:58.696918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=89173a8fed6b4e6381a006abae66baa7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:58.708824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:58.811801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:58.911197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89173a8fed6b4e6381a006abae66baa7', 'request_id': 'req-20be7e2c-1558-4954-8ab5-3f184e66e7fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:58.929363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89173a8fed6b4e6381a006abae66baa7', 'request_id': 'req-20be7e2c-1558-4954-8ab5-3f184e66e7fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:58.947354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89173a8fed6b4e6381a006abae66baa7', 'request_id': 'req-20be7e2c-1558-4954-8ab5-3f184e66e7fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:59.055944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] The token cache is being invalidate because project 89173a8fed6b4e6381a006abae66baa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:13:59.056473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:59.059459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-20be7e2c-1558-4954-8ab5-3f184e66e7fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:13:59.063691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 867/1728] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:13:58 2022] DELETE /identity/v3/projects/89173a8fed6b4e6381a006abae66baa7 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:13:59.068817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3aa14-0338-4efd-937b-d7e7078450ca None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:59.070115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3aa14-0338-4efd-937b-d7e7078450ca None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:59.070115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3aa14-0338-4efd-937b-d7e7078450ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:59.072347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7118e0-0991-4980-9d8a-d4e9ace7a63e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:59.072638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7118e0-0991-4980-9d8a-d4e9ace7a63e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:59.073035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7118e0-0991-4980-9d8a-d4e9ace7a63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:59.133965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-63b3aa14-0338-4efd-937b-d7e7078450ca None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:59.142877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-63b3aa14-0338-4efd-937b-d7e7078450ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:59.184994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5d7118e0-0991-4980-9d8a-d4e9ace7a63e None None] MFA Rules not processed for user `a27f65d219f84286987f1266c88de1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:59.186282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d7118e0-0991-4980-9d8a-d4e9ace7a63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:59.315444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 862/1729] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:59.331408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810f53-274c-4e40-ac8e-15d448e5d230 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:59.337854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810f53-274c-4e40-ac8e-15d448e5d230 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:59.338514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810f53-274c-4e40-ac8e-15d448e5d230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:59.374299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:13:59.406004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 868/1730] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:59.429382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7f810f53-274c-4e40-ac8e-15d448e5d230 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:13:59.440319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7f810f53-274c-4e40-ac8e-15d448e5d230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:13:59.467415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 863/1731] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:13:59.472125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7118e0-0991-4980-9d8a-d4e9ace7a63e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:59.524126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7118e0-0991-4980-9d8a-d4e9ace7a63e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:59.551416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:59.552340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:59.552576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:59.553152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:59.556401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:13:59.557058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:59.558988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:59.589234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 869/1732] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:13:59.593219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f810f53-274c-4e40-ac8e-15d448e5d230 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:59.604398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f810f53-274c-4e40-ac8e-15d448e5d230 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:59.610892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:59.611973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:59.612236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:59.612653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:59.613378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:59.615838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:13:59.794826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 864/1733] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:13:59 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:13:59.799682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:13:59.823158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca07e9c-e189-4faa-842b-e8ed21ba07d8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:13:59.915429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:13:59.916200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:13:59.916721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:13:59.917102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:13:59.918125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:13:59.920176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:00.052358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:00.056027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 870/1734] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:13:59 2022] POST /identity/v3/users => generated 283 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:00.065971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:00.103244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca073796-161a-4de9-8861-2aceb46a44d4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:00.119371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a85e8f-3bfe-4c5c-ac6f-14cf051b5978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:00.131988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a85e8f-3bfe-4c5c-ac6f-14cf051b5978 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:00.132654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a85e8f-3bfe-4c5c-ac6f-14cf051b5978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:00.137813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a85e8f-3bfe-4c5c-ac6f-14cf051b5978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:00.141658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a85e8f-3bfe-4c5c-ac6f-14cf051b5978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:00.146502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a85e8f-3bfe-4c5c-ac6f-14cf051b5978 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:00.189472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 865/1735] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:00.193377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:00.217661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164d42f-352f-4b6b-b84b-778a77132e24 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:00.236074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2ab8-f1ac-423a-ab67-6eefd1159a15 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:00.236718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2ab8-f1ac-423a-ab67-6eefd1159a15 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:00.237170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2ab8-f1ac-423a-ab67-6eefd1159a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:00.244436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:00.245331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:00.246514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:00.246514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:00.249237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:00.259372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:00.284698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 871/1736] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:00.287909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:00.291209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-01bc2ab8-f1ac-423a-ab67-6eefd1159a15 None None] MFA Rules not processed for user `93ef027007184b8180325d6fb26324a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:00.292541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01bc2ab8-f1ac-423a-ab67-6eefd1159a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:00.306316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ff56-f6d0-45c3-af9a-fc757d101338 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:00.312740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:00.314414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:00.315810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:00.322720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] PATH_INFO: `/v3/projects/cb3b32b725ad415fa318aa6da84b3a4c/users/060f0ea0fe7746f1bd4ca8d39add401c/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:00.415036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3b32b725ad415fa318aa6da84b3a4c, user_id=060f0ea0fe7746f1bd4ca8d39add401c, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:00.417790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:00.553542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 866/1737] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Sep 4 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:00.569620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 872/1738] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:00 2022] PUT /identity/v3/projects/cb3b32b725ad415fa318aa6da84b3a4c/users/060f0ea0fe7746f1bd4ca8d39add401c/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:00.580724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081184e5-dfed-49e2-ac49-7a07cb97b8e4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:00.583903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081184e5-dfed-49e2-ac49-7a07cb97b8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:00.584188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081184e5-dfed-49e2-ac49-7a07cb97b8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:00.661466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:00.663731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:00.664480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-081184e5-dfed-49e2-ac49-7a07cb97b8e4 None None] MFA Rules not processed for user `060f0ea0fe7746f1bd4ca8d39add401c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:00.666095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-081184e5-dfed-49e2-ac49-7a07cb97b8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:00.866953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6414-8937-4cb7-b51c-c34275efebb2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:00.882129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:00.882780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:00.883068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:00.883326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:00.897409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 867/1739] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:00.902120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081184e5-dfed-49e2-ac49-7a07cb97b8e4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:00.927498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:00.929070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:00.956123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 873/1740] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:01.091890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081184e5-dfed-49e2-ac49-7a07cb97b8e4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:01.126032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:01.126032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:01.126032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:01.126032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:01.126332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:01.126332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:01.129619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:01.311118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:01.322892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 868/1741] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:00 2022] POST /identity/v3/projects => generated 363 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:01.342760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa357e-c327-4b97-8d40-1929451f30be service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:01.370168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:01.373621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:01.635617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a850f2-aac0-491c-86ee-9b1e01a66c37 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:01.639595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:01.647008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:01.647008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:01.647008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:01.647008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:01.651072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:01.651931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:01.652627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:01.652921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:01.653200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] PATH_INFO: `/v3/projects/e774896864974d3da13f42fcef8004d7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:01.668195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e774896864974d3da13f42fcef8004d7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:01.675753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 16:14:01.675753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Sep 04 16:14:01.675753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: Sep 04 16:14:01.676758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:01.825103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 874/1742] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:01 2022] POST /identity/v3/users => generated 288 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:01.826051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:01.911946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e114ad-1f30-4f92-a4aa-4648aad2c82d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:01.974293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:01.980179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e774896864974d3da13f42fcef8004d7', 'request_id': 'req-98b4a189-70c5-43b6-b3f9-973035cacfd3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:02.088138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:02.088773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.089056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.089334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.090057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:02.091690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:02.108522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e774896864974d3da13f42fcef8004d7', 'request_id': 'req-98b4a189-70c5-43b6-b3f9-973035cacfd3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:02.119669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e774896864974d3da13f42fcef8004d7', 'request_id': 'req-98b4a189-70c5-43b6-b3f9-973035cacfd3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:02.127834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 875/1743] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:02.132598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:02.167432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39aaf9-2c6d-4008-b823-977a039c3997 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:02.167432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:02.167432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.167432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.167850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] PATH_INFO: `/v3/projects/b3f7b832edec4578aec01fb56ab944c3/users/ed17aad886f84fc6940933be69aacfe5/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.188553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] The token cache is being invalidate because project e774896864974d3da13f42fcef8004d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:02.191191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:02.194087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-98b4a189-70c5-43b6-b3f9-973035cacfd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:02.195996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 869/1744] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:01 2022] DELETE /identity/v3/projects/e774896864974d3da13f42fcef8004d7 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:02.203098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a4a9b-8a01-4dc5-85c5-03b294eb15dc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.204378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a4a9b-8a01-4dc5-85c5-03b294eb15dc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.204674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a4a9b-8a01-4dc5-85c5-03b294eb15dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.210281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f7b832edec4578aec01fb56ab944c3, user_id=ed17aad886f84fc6940933be69aacfe5, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:02.214090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:02.290955 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7e1a4a9b-8a01-4dc5-85c5-03b294eb15dc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:02.293136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7e1a4a9b-8a01-4dc5-85c5-03b294eb15dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:02.304215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 870/1745] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:02.318055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967bd53-f04b-4348-a74e-3db7e159a139 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.318923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967bd53-f04b-4348-a74e-3db7e159a139 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.320613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967bd53-f04b-4348-a74e-3db7e159a139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.391233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 876/1746] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:02 2022] PUT /identity/v3/projects/b3f7b832edec4578aec01fb56ab944c3/users/ed17aad886f84fc6940933be69aacfe5/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:02.393182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:02.395267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:02.471412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7967bd53-f04b-4348-a74e-3db7e159a139 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:02.472854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7967bd53-f04b-4348-a74e-3db7e159a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:02.531667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb563bfb-eefd-450b-b3c2-47db569d774b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:02.555079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:02.555824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.556112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.557083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.558880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:02.566994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:02.598290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 877/1747] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:02.599376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:02.642905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 871/1748] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:02.643432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967bd53-f04b-4348-a74e-3db7e159a139 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:02.657117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090fc4-0d77-464b-82c5-0b73f1489198 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:02.664014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967bd53-f04b-4348-a74e-3db7e159a139 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:02.669419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff6481-3774-4440-b8d2-3f222d19e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:02.670552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6481-3774-4440-b8d2-3f222d19e920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.670552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6481-3774-4440-b8d2-3f222d19e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.670745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6481-3774-4440-b8d2-3f222d19e920 admin admin] PATH_INFO: `/v3/projects/b3f7b832edec4578aec01fb56ab944c3/users/ed17aad886f84fc6940933be69aacfe5/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.681815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff6481-3774-4440-b8d2-3f222d19e920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f7b832edec4578aec01fb56ab944c3, user_id=ed17aad886f84fc6940933be69aacfe5, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:02.683313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:02.684142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.684515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff6481-3774-4440-b8d2-3f222d19e920 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:02.686263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.686723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.687580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:02.690341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:02.693166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:02.708571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 872/1749] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:02.711034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:02.714125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 878/1750] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:02 2022] PUT /identity/v3/projects/b3f7b832edec4578aec01fb56ab944c3/users/ed17aad886f84fc6940933be69aacfe5/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:02.716380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:02.727848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.728143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.728621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.747865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df54f4-c983-433d-8598-029b007082d1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:02.800752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] MFA Rules not processed for user `ed17aad886f84fc6940933be69aacfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:02.803035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:02.940650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:02.941435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:02.941848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:02.942216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:02.943572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:02.950888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:02.963272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 879/1751] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:02.977907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:02.980640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:02.994035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:03.142649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01a528-232c-415d-a750-d84b685927f9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:03.156192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:03.159906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:03.162592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:03.167365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:03.191187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:03.195450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:03.225629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 880/1752] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:03.375067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 873/1753] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:02 2022] POST /identity/v3/projects => generated 375 bytes in 660 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:03.389394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:03.427531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ae621-7994-43c4-b7cc-32f029b13afa service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:03.624661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:03.686389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:03.687105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:03.687453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:03.688256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:03.689526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:03.702394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:03.867973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 881/1754] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:03 2022] POST /identity/v3/users => generated 288 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:03.870201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:03.872146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f039b8-87e1-4393-bd5f-086b7fbf9c24 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:03.881975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cd5fb-3610-4025-ad10-1787983525c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:03.884892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cd5fb-3610-4025-ad10-1787983525c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:03.885434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cd5fb-3610-4025-ad10-1787983525c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:03.885755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cd5fb-3610-4025-ad10-1787983525c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:03.902927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-914cd5fb-3610-4025-ad10-1787983525c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:03.912024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c6f3b-d0cc-4933-ad53-2eff8eaa4c59 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:03.952559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:03.953257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:03.954359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:03.962704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:03.972373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:03.972373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:04.052996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cd5fb-3610-4025-ad10-1787983525c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:04.056535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cd5fb-3610-4025-ad10-1787983525c8 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:04.059685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 882/1755] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:04.063839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:04.083299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 874/1756] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:04.114277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8364cd-0e99-454e-8aa3-5f9a8f715586 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:04.137041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16f403-2e41-4b00-976a-5a72bc661fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:04.137717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f403-2e41-4b00-976a-5a72bc661fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:04.138002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f403-2e41-4b00-976a-5a72bc661fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:04.138277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f403-2e41-4b00-976a-5a72bc661fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:04.138939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16f403-2e41-4b00-976a-5a72bc661fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:04.140584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16f403-2e41-4b00-976a-5a72bc661fc1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:04.148898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5217-066c-40b4-a8db-15dfbfd1710a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:04.149543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5217-066c-40b4-a8db-15dfbfd1710a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:04.150064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5217-066c-40b4-a8db-15dfbfd1710a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:04.152307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 875/1757] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:14:04.172739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 883/1758] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:04.176616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c5217-066c-40b4-a8db-15dfbfd1710a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:04.198462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c5217-066c-40b4-a8db-15dfbfd1710a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:04.205083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73ce9e-c6b0-4b11-bdfe-e212e874757c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:04.205083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73ce9e-c6b0-4b11-bdfe-e212e874757c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:04.205083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73ce9e-c6b0-4b11-bdfe-e212e874757c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:04.225530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b0d57-bb23-4de6-bd9a-1ace3da68927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:04.227398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0d57-bb23-4de6-bd9a-1ace3da68927 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:04.230187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0d57-bb23-4de6-bd9a-1ace3da68927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:04.230604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0d57-bb23-4de6-bd9a-1ace3da68927 admin admin] PATH_INFO: `/v3/projects/84f875c138cc4d3c915351cbbf58f559/users/82d101b28c5448248a03ab4d370a6602/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:04.246285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8a73ce9e-c6b0-4b11-bdfe-e212e874757c None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:04.249594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8a73ce9e-c6b0-4b11-bdfe-e212e874757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:04.271276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b0d57-bb23-4de6-bd9a-1ace3da68927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f875c138cc4d3c915351cbbf58f559, user_id=82d101b28c5448248a03ab4d370a6602, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:04.303593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b0d57-bb23-4de6-bd9a-1ace3da68927 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:04.440935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 876/1759] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:04 2022] PUT /identity/v3/projects/84f875c138cc4d3c915351cbbf58f559/users/82d101b28c5448248a03ab4d370a6602/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:04.451752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9810f-5f8e-41fc-8b1c-65697531f66e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:04.452460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9810f-5f8e-41fc-8b1c-65697531f66e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:04.453181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9810f-5f8e-41fc-8b1c-65697531f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:04.469819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 884/1760] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:04.479070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a0003-8d18-40d2-9ea1-ab757c6283b3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:04.479363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a0003-8d18-40d2-9ea1-ab757c6283b3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:04.479645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a0003-8d18-40d2-9ea1-ab757c6283b3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:04.496186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 885/1761] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:14:04 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:14:04.531031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-61a9810f-5f8e-41fc-8b1c-65697531f66e None None] MFA Rules not processed for user `82d101b28c5448248a03ab4d370a6602`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:04.532584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-61a9810f-5f8e-41fc-8b1c-65697531f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:04.548643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:04.549225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:04.550290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:04.652845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:04.678544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:04.679889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:04.817107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:04.824193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 877/1762] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:04.830149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9810f-5f8e-41fc-8b1c-65697531f66e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:04.904743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 886/1763] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:05.007255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9810f-5f8e-41fc-8b1c-65697531f66e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:05.021635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:05.023012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:05.023309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:05.023593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:05.024262 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:05.030917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:05.123994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:05.126597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:05.144826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319775cd-c2ee-423d-83cd-8692b212bbee None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:05.152891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:05.156763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:05.168714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:05.176206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:05.185055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 878/1764] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:04 2022] POST /identity/v3/projects => generated 377 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:05.188682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:05.221196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:05.222987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:05.281888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 887/1765] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:05.287564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:05.289292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:05.486610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f474b2c-2641-4b44-9803-3e199136b721 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:05.611476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5f24a-4e7e-44f7-aeee-c3f4389b4b72 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:05.630687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:05.633771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:05.633771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:05.633771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:05.633771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:05.662946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:05.676680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71438a-c1f8-4a43-827d-4ada9e91dded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:05.677321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71438a-c1f8-4a43-827d-4ada9e91dded service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:05.677628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71438a-c1f8-4a43-827d-4ada9e91dded service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:05.677942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71438a-c1f8-4a43-827d-4ada9e91dded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:05.710224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71438a-c1f8-4a43-827d-4ada9e91dded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:05.716588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71438a-c1f8-4a43-827d-4ada9e91dded service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:05.750604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 888/1766] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:05.752344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512eb28-5c99-47c4-babf-43575d5318b2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:05.752924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512eb28-5c99-47c4-babf-43575d5318b2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:05.752924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512eb28-5c99-47c4-babf-43575d5318b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:05.800495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f512eb28-5c99-47c4-babf-43575d5318b2 None None] MFA Rules not processed for user `6e5c22b631df4600b9f920120b67fc67`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:05.801956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f512eb28-5c99-47c4-babf-43575d5318b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:05.851120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:05.867576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 879/1767] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:05 2022] POST /identity/v3/users => generated 295 bytes in 676 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:05.870767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:05.880142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:05.919269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef926a-c346-41c4-8a60-90b06eede0e0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:05.947063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:05.948573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:05.950072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:05.950303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] PATH_INFO: `/v3/users/9e563e3bbd07468f9c7a29837a10de0c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:05.961808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 889/1768] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:05.962373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e563e3bbd07468f9c7a29837a10de0c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:05.962463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512eb28-5c99-47c4-babf-43575d5318b2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:05.962535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:06.003845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512eb28-5c99-47c4-babf-43575d5318b2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:06.019055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:06.030019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:06.031421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:06.033596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:06.042110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:06.044127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:06.048685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:06.088109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 890/1769] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:06.089750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:06.131534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8f446-ab80-4484-b8b8-bbd007ecc5da admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:06.162770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:06.163612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:06.163992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:06.164386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] PATH_INFO: `/v3/projects/c1136f95971e4e1983300e81b5d469c4/users/e7fbb148c2344dcabd2063b60c841e12/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:06.219424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1136f95971e4e1983300e81b5d469c4, user_id=e7fbb148c2344dcabd2063b60c841e12, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:06.228072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:06.317970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e563e3bbd07468f9c7a29837a10de0c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:06.319780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e563e3bbd07468f9c7a29837a10de0c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:06.347160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 891/1770] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:06 2022] PUT /identity/v3/projects/c1136f95971e4e1983300e81b5d469c4/users/e7fbb148c2344dcabd2063b60c841e12/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:06.380970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:06.390079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:06.442047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e563e3bbd07468f9c7a29837a10de0c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:06.443482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e563e3bbd07468f9c7a29837a10de0c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:06.480826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b99357fa-c04f-4a6a-b51c-62ac3a12c765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e563e3bbd07468f9c7a29837a10de0c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:06.533368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 880/1771] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:05 2022] DELETE /identity/v3/users/9e563e3bbd07468f9c7a29837a10de0c => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:06.537719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:06.538446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:06.542791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:06.595631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:06.600007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:06.690890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9f3a3-7abe-4992-8223-b6528a276c2b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:06.748752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 881/1772] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:06.847269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:06.847269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:06.847269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:06.847269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:06.848401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:06.850405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:06.850540 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:06.853071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:06.888320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 892/1773] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 536 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:06.895151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:06.927491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bb492-d0cc-4b52-a169-12b177fe05a3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:06.947808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d1b12-90d3-4c93-950a-f481df811b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:06.958416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d1b12-90d3-4c93-950a-f481df811b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:06.958741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d1b12-90d3-4c93-950a-f481df811b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:06.959004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d1b12-90d3-4c93-950a-f481df811b5f admin admin] PATH_INFO: `/v3/projects/c1136f95971e4e1983300e81b5d469c4/users/e7fbb148c2344dcabd2063b60c841e12/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:06.961343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d1b12-90d3-4c93-950a-f481df811b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1136f95971e4e1983300e81b5d469c4, user_id=e7fbb148c2344dcabd2063b60c841e12, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:06.964105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d1b12-90d3-4c93-950a-f481df811b5f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:06.981294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 893/1774] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:06 2022] PUT /identity/v3/projects/c1136f95971e4e1983300e81b5d469c4/users/e7fbb148c2344dcabd2063b60c841e12/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:06.993927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:06.994323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:06.998130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:07.015383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbc0d8-5a5a-47a0-8760-d7ba92c8215d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:07.066192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:07.077086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] MFA Rules not processed for user `e7fbb148c2344dcabd2063b60c841e12`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:07.079225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:07.149833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:07.150584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:07.150930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:07.151372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:07.348090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:07.351469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:07.373298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:07.399154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 882/1775] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:07.420530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 894/1776] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:07.438340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:07.441044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:07.618301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1364-626b-4922-a892-12c43c3a59e0 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:07.628302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:07.629207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:07.629725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:07.630157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:07.633988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:07.775744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:07.777326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:07.805093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 883/1777] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:08.577304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:08.579995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:08.632023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566306bc-f565-4cb9-be2c-53404e080b42 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:08.640146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:08.642328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:08.642981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:08.643476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] PATH_INFO: `/v3/projects/328290a36e0f451c9117bcce81c7b753` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:08.646427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=328290a36e0f451c9117bcce81c7b753)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:08.648906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:08.825491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328290a36e0f451c9117bcce81c7b753', 'request_id': 'req-7a273031-05e8-4aea-9c73-9b8efd071101'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:08.846431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328290a36e0f451c9117bcce81c7b753', 'request_id': 'req-7a273031-05e8-4aea-9c73-9b8efd071101'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:08.856891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328290a36e0f451c9117bcce81c7b753', 'request_id': 'req-7a273031-05e8-4aea-9c73-9b8efd071101'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:09.508834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] The token cache is being invalidate because project 328290a36e0f451c9117bcce81c7b753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:09.509313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:09.511001 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:09.517869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 895/1778] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:08 2022] DELETE /identity/v3/projects/328290a36e0f451c9117bcce81c7b753 => generated 0 bytes in 941 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:09.521655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:09.541622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffcc51-3737-4895-963d-cf916d5a2778 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:09.636091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:09.639057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:09.664999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:09.692448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:09.699412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:09.699412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:09.699412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] PATH_INFO: `/v3/users/f7fbe9450cc445abb13d2e7ca654a099` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:09.699412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fbe9450cc445abb13d2e7ca654a099)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:09.699412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:09.763413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a273031-05e8-4aea-9c73-9b8efd071101 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:09.774901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:09.775967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:09.776486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:09.776986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] PATH_INFO: `/v3/users/bf729c8e202544d4b4000e8adc5a0da3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:09.780802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf729c8e202544d4b4000e8adc5a0da3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:09.784334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:09.841633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:09.928077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fbe9450cc445abb13d2e7ca654a099'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:09.930390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fbe9450cc445abb13d2e7ca654a099'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.058653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fbe9450cc445abb13d2e7ca654a099'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.058653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fbe9450cc445abb13d2e7ca654a099'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.082767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fbe9450cc445abb13d2e7ca654a099'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.107413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 884/1779] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:09 2022] DELETE /identity/v3/users/f7fbe9450cc445abb13d2e7ca654a099 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:10.108454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:10.110657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:10.144957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf729c8e202544d4b4000e8adc5a0da3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.144957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf729c8e202544d4b4000e8adc5a0da3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.188817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf729c8e202544d4b4000e8adc5a0da3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.215785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552471f0-3e3a-4f30-886a-72a71b43b5dd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:10.223147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:10.223897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:10.224231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:10.224612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] PATH_INFO: `/v3/users/060f0ea0fe7746f1bd4ca8d39add401c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:10.227507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060f0ea0fe7746f1bd4ca8d39add401c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:10.230435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:10.282412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf729c8e202544d4b4000e8adc5a0da3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.283064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a4f34e0-3454-4dbb-a75f-321e8cf4808c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf729c8e202544d4b4000e8adc5a0da3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.305962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 896/1780] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:09 2022] DELETE /identity/v3/users/bf729c8e202544d4b4000e8adc5a0da3 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:10.552468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060f0ea0fe7746f1bd4ca8d39add401c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.554312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060f0ea0fe7746f1bd4ca8d39add401c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.672297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060f0ea0fe7746f1bd4ca8d39add401c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.673188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060f0ea0fe7746f1bd4ca8d39add401c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.699415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060f0ea0fe7746f1bd4ca8d39add401c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:10.707142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:10.730997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 885/1781] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:10 2022] DELETE /identity/v3/users/060f0ea0fe7746f1bd4ca8d39add401c => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:10.734555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:10.734555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:10.734555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:10.753842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:10.755444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:10.821436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:10.823018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:10.853028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 897/1782] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:10.878057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:10.879695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:11.011099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b4ab-136a-4db0-92fe-b763ebece079 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:11.019225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:11.019861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:11.020150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:11.020430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:11.039023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655901a1-a628-4f14-af03-35e21d6e9883 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:11.047722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:11.048603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:11.049063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:11.049376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] PATH_INFO: `/v3/projects/444b2403d08a4782a56a5543d6c50029` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:11.050773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444b2403d08a4782a56a5543d6c50029)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:11.053078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:11.064215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:11.066817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:11.097781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 898/1783] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:11.215118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:11.217079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:11.233354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc2708-0cf3-4953-88a5-b99edcbabe01 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:11.241474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:11.242586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:11.243545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:11.244256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] PATH_INFO: `/v3/projects/d5679ab32e2746c399563a107655d4e8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:11.247560 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5679ab32e2746c399563a107655d4e8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:11.269278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:11.320198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444b2403d08a4782a56a5543d6c50029', 'request_id': 'req-74176fb3-81bf-415b-ae54-3ae7b9458cc5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.393820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444b2403d08a4782a56a5543d6c50029', 'request_id': 'req-74176fb3-81bf-415b-ae54-3ae7b9458cc5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.400508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444b2403d08a4782a56a5543d6c50029', 'request_id': 'req-74176fb3-81bf-415b-ae54-3ae7b9458cc5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.415398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] The token cache is being invalidate because project 444b2403d08a4782a56a5543d6c50029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:11.415794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.416735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.421405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 886/1784] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:10 2022] DELETE /identity/v3/projects/444b2403d08a4782a56a5543d6c50029 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:11.423339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:11.424798 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:11.528147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74176fb3-81bf-415b-ae54-3ae7b9458cc5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:11.551977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5679ab32e2746c399563a107655d4e8', 'request_id': 'req-60f83584-0137-4142-8868-568363f25a25'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.568085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5679ab32e2746c399563a107655d4e8', 'request_id': 'req-60f83584-0137-4142-8868-568363f25a25'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.590471 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5679ab32e2746c399563a107655d4e8', 'request_id': 'req-60f83584-0137-4142-8868-568363f25a25'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.642858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:11.643593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:11.644095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:11.644456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] PATH_INFO: `/v3/users/a27f65d219f84286987f1266c88de1ea` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:11.645741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a27f65d219f84286987f1266c88de1ea)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:11.654231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:11.675063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:11.717604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] The token cache is being invalidate because project d5679ab32e2746c399563a107655d4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:11.718212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.719864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60f83584-0137-4142-8868-568363f25a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.723180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 899/1785] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:11 2022] DELETE /identity/v3/projects/d5679ab32e2746c399563a107655d4e8 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:11.733867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9734-e15a-4d07-bba6-bee8431d7e91 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:11.735256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9734-e15a-4d07-bba6-bee8431d7e91 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:11.736118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9734-e15a-4d07-bba6-bee8431d7e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:11.816172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1abd9734-e15a-4d07-bba6-bee8431d7e91 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:11.818608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1abd9734-e15a-4d07-bba6-bee8431d7e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:11.872556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 900/1786] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:11.872556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d497-53b6-4d03-959c-060adfc93b88 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:11.872556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d497-53b6-4d03-959c-060adfc93b88 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:11.872556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d497-53b6-4d03-959c-060adfc93b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:11.923416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-83a9d497-53b6-4d03-959c-060adfc93b88 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:11.925610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-83a9d497-53b6-4d03-959c-060adfc93b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:11.945414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a27f65d219f84286987f1266c88de1ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.946633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a27f65d219f84286987f1266c88de1ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:11.973794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 901/1787] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:11.975379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9d497-53b6-4d03-959c-060adfc93b88 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:11.985487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9d497-53b6-4d03-959c-060adfc93b88 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:11.990432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:11.991051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:11.991355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:11.991629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:11.992546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:11.992849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:11.994482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:12.024636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 902/1788] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:12.026320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:12.036438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a27f65d219f84286987f1266c88de1ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.037174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a27f65d219f84286987f1266c88de1ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.044011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddb86f-dcb1-432a-90ce-d78b86c3f25b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:12.052198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193b79e-620d-44f0-9b26-9d9747d37671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:12.053745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193b79e-620d-44f0-9b26-9d9747d37671 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.054294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193b79e-620d-44f0-9b26-9d9747d37671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.058199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193b79e-620d-44f0-9b26-9d9747d37671 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:12.058199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193b79e-620d-44f0-9b26-9d9747d37671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:12.058199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193b79e-620d-44f0-9b26-9d9747d37671 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:12.063959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a27f65d219f84286987f1266c88de1ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.100793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 887/1789] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:11 2022] DELETE /identity/v3/users/a27f65d219f84286987f1266c88de1ea => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:12.167296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 903/1790] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:12 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:12.168444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:12.319717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aba520-ee71-41f3-836f-2bd1658cdd3b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:12.469762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8f5f-7805-403b-bd08-5bea2bd567a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:12.470578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8f5f-7805-403b-bd08-5bea2bd567a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.470698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8f5f-7805-403b-bd08-5bea2bd567a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.475249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8f5f-7805-403b-bd08-5bea2bd567a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:12.475249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8f5f-7805-403b-bd08-5bea2bd567a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:12.475249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8f5f-7805-403b-bd08-5bea2bd567a7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:12.490653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:12.509775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545804bf-4911-4e86-9919-7880de210118 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.509897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545804bf-4911-4e86-9919-7880de210118 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.510323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545804bf-4911-4e86-9919-7880de210118 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:12.512226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 904/1791] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:14:12.527840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545804bf-4911-4e86-9919-7880de210118 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:12.535041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-545804bf-4911-4e86-9919-7880de210118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:12.586662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545804bf-4911-4e86-9919-7880de210118 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:12.592067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:12.592832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.593196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.593610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] PATH_INFO: `/v3/projects/689b750af589492eb8ffef7969a92282` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:12.594967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689b750af589492eb8ffef7969a92282)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:12.596875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:12.634615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 888/1792] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:12 2022] POST /identity/v3/users => generated 291 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:12.636208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df545226-3358-4b06-be2e-2fbd5a97bcc7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.637015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df545226-3358-4b06-be2e-2fbd5a97bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.638242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df545226-3358-4b06-be2e-2fbd5a97bcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:12.673215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-df545226-3358-4b06-be2e-2fbd5a97bcc7 None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:12.674688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-df545226-3358-4b06-be2e-2fbd5a97bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:12.727745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689b750af589492eb8ffef7969a92282', 'request_id': 'req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.735119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689b750af589492eb8ffef7969a92282', 'request_id': 'req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.742970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689b750af589492eb8ffef7969a92282', 'request_id': 'req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.756081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 889/1793] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:12.759116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df545226-3358-4b06-be2e-2fbd5a97bcc7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:12.769761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df545226-3358-4b06-be2e-2fbd5a97bcc7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:12.834366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:12.835084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.835367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.835673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:12.836323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:12.838228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:12.851855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 890/1794] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:12.852987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:12.856368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:12.856949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] The token cache is being invalidate because project 689b750af589492eb8ffef7969a92282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:12.857395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.858322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:12.863123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 905/1795] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:12 2022] DELETE /identity/v3/projects/689b750af589492eb8ffef7969a92282 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:12.865453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:12.868305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:12.874007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20b3ce-f614-4e9a-98d9-8c1e7507731e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:12.881917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:12.882569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.883094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.883094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] PATH_INFO: `/v3/projects/cb3b32b725ad415fa318aa6da84b3a4c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:12.885826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3b32b725ad415fa318aa6da84b3a4c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:12.888309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:12.990572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a7d46-b83b-4d0d-92fa-5a74fc51e340 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:12.998071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:12.998764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:12.999042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:12.999319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.002066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.012199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3b32b725ad415fa318aa6da84b3a4c', 'request_id': 'req-a3b5706d-7afe-461a-91a7-b3cb2296fd13'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.015118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.017880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.028966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 906/1796] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:13.029908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.031948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.062316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2c61-d272-439d-8510-841ec25b10af service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.076619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3b32b725ad415fa318aa6da84b3a4c', 'request_id': 'req-a3b5706d-7afe-461a-91a7-b3cb2296fd13'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.088550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3b32b725ad415fa318aa6da84b3a4c', 'request_id': 'req-a3b5706d-7afe-461a-91a7-b3cb2296fd13'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.107102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] The token cache is being invalidate because project cb3b32b725ad415fa318aa6da84b3a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:13.108291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.109382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a3b5706d-7afe-461a-91a7-b3cb2296fd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.115186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 891/1797] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:12 2022] DELETE /identity/v3/projects/cb3b32b725ad415fa318aa6da84b3a4c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:13.116622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.120360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.122241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.144159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.144159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.144159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.144159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.144506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.144855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.151820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:13.153113 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.157363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 907/1798] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:13.158420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.160479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.176186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 892/1799] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:13.176186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.176721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f060-48e1-4271-9217-e4a354c07a53 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.181663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.181793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.181793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.181793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.181793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] PATH_INFO: `/v3/users/ed17aad886f84fc6940933be69aacfe5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.183273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed17aad886f84fc6940933be69aacfe5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.185121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.189974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca676b95-4ea9-4bb9-b961-fb72149012ad None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.194963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c2f43-022d-42d6-9cbd-e5536f850a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.195417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2f43-022d-42d6-9cbd-e5536f850a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.195701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2f43-022d-42d6-9cbd-e5536f850a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.195971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2f43-022d-42d6-9cbd-e5536f850a23 admin admin] PATH_INFO: `/v3/projects/c08d9900ee684aa28ecad428bc9294f5/users/decf9291a7094a0b8d7f78f81a2ced3d/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.211736 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c2f43-022d-42d6-9cbd-e5536f850a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08d9900ee684aa28ecad428bc9294f5, user_id=decf9291a7094a0b8d7f78f81a2ced3d, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.214621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c2f43-022d-42d6-9cbd-e5536f850a23 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.345846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 893/1800] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:13 2022] PUT /identity/v3/projects/c08d9900ee684aa28ecad428bc9294f5/users/decf9291a7094a0b8d7f78f81a2ced3d/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:13.347490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49626-0b88-420d-80b9-12a8886f8363 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.347861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49626-0b88-420d-80b9-12a8886f8363 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.348306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49626-0b88-420d-80b9-12a8886f8363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.371115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c3d49626-0b88-420d-80b9-12a8886f8363 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:13.372394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c3d49626-0b88-420d-80b9-12a8886f8363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.443330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 894/1801] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:13.444718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.445015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.445297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.479507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed17aad886f84fc6940933be69aacfe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.481326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed17aad886f84fc6940933be69aacfe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.493626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] MFA Rules not processed for user `decf9291a7094a0b8d7f78f81a2ced3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:13.496717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.503147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed17aad886f84fc6940933be69aacfe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.575491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 895/1802] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:13.575491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.579240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.585073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed17aad886f84fc6940933be69aacfe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.588596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed17aad886f84fc6940933be69aacfe5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:13.604274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4af61-c64d-4734-8dd0-30e82f049f1b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.611667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 908/1803] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:13 2022] DELETE /identity/v3/users/ed17aad886f84fc6940933be69aacfe5 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:13.611667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.611667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:13.688663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:13.694780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.695509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.696014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.696708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.697751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:13.698156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.700017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.709740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7a2b2-dfec-4213-8d76-0aa6b980c043 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.716553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.721245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 896/1804] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:13.721387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.721387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.721387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.721387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.721387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.721809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.731553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96644-e1e1-4046-acc9-4f953dc3785d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.736959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.737580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.737876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.738152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.738823 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.740687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.837173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 909/1805] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:13 2022] POST /identity/v3/projects => generated 381 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:13.837586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 897/1806] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:13 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:13.841450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.842080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:13.909191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4858b8-97a5-4848-9650-b4023f2ec612 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.913410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a03aa-46cf-4aa2-815d-fceb55183cef admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:13.915224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.916017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.916347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.916714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.918502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.918815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:13.919521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:13.919935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:13.920361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:13.920996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:13.921189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:13.923852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.030474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 910/1807] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:13 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:14.035016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.045262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa9163-3485-434e-acda-295928a60b82 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.050535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.051127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.051491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.052104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.053378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.055491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.063101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 898/1808] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:13 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:14.067843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 911/1809] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:14.068772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.069037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.070904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.085504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b3c2-d3b4-448a-96e4-844efcb26537 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.088617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd736f-30cb-4876-878a-6ebfef240abf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.090708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.091341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.091961 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.091961 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.092601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.094870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.095710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.095797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.095991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.096266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] PATH_INFO: `/v3/projects/b3f7b832edec4578aec01fb56ab944c3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.097422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f7b832edec4578aec01fb56ab944c3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.099559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.106244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 912/1810] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:14.107193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.118732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619fdf0-e988-44ba-9fe5-6edbf2f6c5a7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.123714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.124310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.124592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.124870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.125844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.127463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.138107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 913/1811] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:14.138886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.148282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0765-62ec-4f96-a16d-cd000b3604cc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.153073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.153669 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.153956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.154267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] PATH_INFO: `/v3/projects/effcdf1a12fd462f986d47992eb60083/users/f728e4be7dbc43779d75e9df5f504d93/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.170007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=effcdf1a12fd462f986d47992eb60083, user_id=f728e4be7dbc43779d75e9df5f504d93, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.173605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.283607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 914/1812] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:14 2022] PUT /identity/v3/projects/effcdf1a12fd462f986d47992eb60083/users/f728e4be7dbc43779d75e9df5f504d93/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:14.284441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.291762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f7b832edec4578aec01fb56ab944c3', 'request_id': 'req-13bef32c-2329-4004-96fc-f6f81796ce4f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:14.356135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f7b832edec4578aec01fb56ab944c3', 'request_id': 'req-13bef32c-2329-4004-96fc-f6f81796ce4f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:14.356306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788b515-3a2f-47d1-a02d-6f9274bde604 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.363270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f7b832edec4578aec01fb56ab944c3', 'request_id': 'req-13bef32c-2329-4004-96fc-f6f81796ce4f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:14.374941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] The token cache is being invalidate because project b3f7b832edec4578aec01fb56ab944c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:14.375234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:14.375907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:14.377450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 899/1813] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:14 2022] DELETE /identity/v3/projects/b3f7b832edec4578aec01fb56ab944c3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:14.378365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.379667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.412887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52b5e6-56ae-40b6-ada3-652679102edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.413468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b5e6-56ae-40b6-ada3-652679102edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.413781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b5e6-56ae-40b6-ada3-652679102edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.414055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b5e6-56ae-40b6-ada3-652679102edf admin admin] PATH_INFO: `/v3/projects/660b5fc09131429e9fcb9d3486cbfe92/users/d8c25a0d720241a597b594f4de2f829a/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.428662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52b5e6-56ae-40b6-ada3-652679102edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=660b5fc09131429e9fcb9d3486cbfe92, user_id=d8c25a0d720241a597b594f4de2f829a, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.431374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52b5e6-56ae-40b6-ada3-652679102edf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.449066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bef32c-2329-4004-96fc-f6f81796ce4f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.458065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3c21f-37ef-4cbd-8d95-cfc0ab266d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.458065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c21f-37ef-4cbd-8d95-cfc0ab266d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.458285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c21f-37ef-4cbd-8d95-cfc0ab266d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.458507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c21f-37ef-4cbd-8d95-cfc0ab266d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.459154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3c21f-37ef-4cbd-8d95-cfc0ab266d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.460784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3c21f-37ef-4cbd-8d95-cfc0ab266d14 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.475314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 900/1814] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:14.475314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0105d70-3bac-435a-b0cb-04c801e7cce9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.475314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0105d70-3bac-435a-b0cb-04c801e7cce9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.475314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0105d70-3bac-435a-b0cb-04c801e7cce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.494706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d0105d70-3bac-435a-b0cb-04c801e7cce9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:14.495915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d0105d70-3bac-435a-b0cb-04c801e7cce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.503832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 901/1815] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:14.504589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0105d70-3bac-435a-b0cb-04c801e7cce9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.515281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0105d70-3bac-435a-b0cb-04c801e7cce9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.518800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 915/1816] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:14 2022] PUT /identity/v3/projects/660b5fc09131429e9fcb9d3486cbfe92/users/d8c25a0d720241a597b594f4de2f829a/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:14.520296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35803fc4-cfea-451b-8b80-aacaa2113a37 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.520398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35803fc4-cfea-451b-8b80-aacaa2113a37 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.520646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35803fc4-cfea-451b-8b80-aacaa2113a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.542192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-35803fc4-cfea-451b-8b80-aacaa2113a37 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:14.543434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-35803fc4-cfea-451b-8b80-aacaa2113a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.590479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.591155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.591438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.591712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] PATH_INFO: `/v3/projects/effcdf1a12fd462f986d47992eb60083/users/f728e4be7dbc43779d75e9df5f504d93/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.593830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=effcdf1a12fd462f986d47992eb60083, user_id=f728e4be7dbc43779d75e9df5f504d93, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.596497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.608951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 916/1817] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:14.608951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3428d1-c094-4053-b6e0-44a4e64661df None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.608951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3428d1-c094-4053-b6e0-44a4e64661df None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.608951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3428d1-c094-4053-b6e0-44a4e64661df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.627672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 902/1818] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:14 2022] PUT /identity/v3/projects/effcdf1a12fd462f986d47992eb60083/users/f728e4be7dbc43779d75e9df5f504d93/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:14.628750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.631433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:14.644813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2a3428d1-c094-4053-b6e0-44a4e64661df None None] MFA Rules not processed for user `d8c25a0d720241a597b594f4de2f829a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:14.646344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2a3428d1-c094-4053-b6e0-44a4e64661df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.709826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a5bb0-b317-46ec-9023-3b0a43f7da1a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.722795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.725717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.725717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.725717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.725717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:14.725717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.727010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.736011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 917/1819] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:14.737922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8483-c0df-4c85-8bf2-612aeb8c5f64 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.738533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8483-c0df-4c85-8bf2-612aeb8c5f64 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.738956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8483-c0df-4c85-8bf2-612aeb8c5f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.739885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 903/1820] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:14.740574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.744289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.757084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9f8e6-b9ed-4b55-9906-79936b5611a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.762069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.762675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.762958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.763231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.763863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.766038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:14.775924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e44d8483-c0df-4c85-8bf2-612aeb8c5f64 None None] MFA Rules not processed for user `f728e4be7dbc43779d75e9df5f504d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:14.777278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e44d8483-c0df-4c85-8bf2-612aeb8c5f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.849900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 918/1821] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:14.850825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d8483-c0df-4c85-8bf2-612aeb8c5f64 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.866182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d8483-c0df-4c85-8bf2-612aeb8c5f64 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.866725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 904/1822] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:14 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:14.867479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:14.871467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:14.948895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76fc3-f1a0-46d4-885b-261eb78bc28f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:14.952505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.953173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.953812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.953812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.956889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:14.957644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:14.958194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:14.958340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:14.959135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:14.961162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.050818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.052291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.063632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 905/1823] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:15.064471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.081894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 919/1824] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:14 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:15.083152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.084979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd7f84-f7cf-4898-a396-b28b1c99da92 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.183410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.184333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.184480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.184908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.185868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.194280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.202592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a32eb-5389-4198-83ad-41d3fefeaa62 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.216775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.217537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.217926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.218303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.220136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.233613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.252000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:15.324323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 906/1825] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:15 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:15.334917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.350573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244edcff-da39-4f58-a387-74dc6c5a216c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.354292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 920/1826] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:15 2022] POST /identity/v3/users => generated 294 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:15.355513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.360986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.362377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.362867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.363319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.364332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.366589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.384004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 907/1827] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:15.387044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.395021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d3cad-b645-4e95-bfbd-5a42366888f5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.402822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.403140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.403239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.403581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.404310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.407208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.411323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fba62-cd12-45c5-a78e-95dff4dfa8cf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.429113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 921/1828] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:15.430702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.431755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.432629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.433060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.433494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] PATH_INFO: `/v3/projects/d8054646d50741489dc193639b4c80e9/users/857e664f7fd94ffea53cd0d07dba2c4d/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.443258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7f8b8-d20b-4de7-be83-c74eccb7c26b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.448606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.449520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.450097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.450488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.451234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.454914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.481313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 922/1829] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:15.486228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.500566 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8054646d50741489dc193639b4c80e9, user_id=857e664f7fd94ffea53cd0d07dba2c4d, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.503996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.508034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a77b4e-ffb2-4350-9912-83aa8ca54287 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.514146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f2162-efab-4b38-a9f2-43a58d9638e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.514976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2162-efab-4b38-a9f2-43a58d9638e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.515260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2162-efab-4b38-a9f2-43a58d9638e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.515674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2162-efab-4b38-a9f2-43a58d9638e5 admin admin] PATH_INFO: `/v3/projects/7a717494cd4a4526aace4b5563254943/users/3d98974d9aca46e09afa2a6b163c311f/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.549597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f2162-efab-4b38-a9f2-43a58d9638e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a717494cd4a4526aace4b5563254943, user_id=3d98974d9aca46e09afa2a6b163c311f, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.552309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f2162-efab-4b38-a9f2-43a58d9638e5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.629134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 908/1830] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:15 2022] PUT /identity/v3/projects/d8054646d50741489dc193639b4c80e9/users/857e664f7fd94ffea53cd0d07dba2c4d/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:15.633552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.665655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 923/1831] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:15 2022] PUT /identity/v3/projects/7a717494cd4a4526aace4b5563254943/users/3d98974d9aca46e09afa2a6b163c311f/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:15.679306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bd68b-df8f-48f9-a24c-afa10eb56da0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.679625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bd68b-df8f-48f9-a24c-afa10eb56da0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.679904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bd68b-df8f-48f9-a24c-afa10eb56da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.726656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5d5bd68b-df8f-48f9-a24c-afa10eb56da0 None None] MFA Rules not processed for user `3d98974d9aca46e09afa2a6b163c311f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:15.727902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d5bd68b-df8f-48f9-a24c-afa10eb56da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:15.751469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88ca0e-6b57-4d1f-9dcf-1788a8203756 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.819544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.821354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.821658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.821938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.822592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.825083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.841173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 909/1832] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:15.841597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 924/1833] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:15.846622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.849424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bd68b-df8f-48f9-a24c-afa10eb56da0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:15.860820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0f01-22b7-49ef-ab9f-f53c9c6d00a0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.865249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bd68b-df8f-48f9-a24c-afa10eb56da0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:15.867073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca49a0-1f22-4e54-881b-20d928cb8eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.868069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca49a0-1f22-4e54-881b-20d928cb8eec admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.868546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca49a0-1f22-4e54-881b-20d928cb8eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.868990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca49a0-1f22-4e54-881b-20d928cb8eec admin admin] PATH_INFO: `/v3/projects/d8054646d50741489dc193639b4c80e9/users/857e664f7fd94ffea53cd0d07dba2c4d/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.890251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:15.891519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.892246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca49a0-1f22-4e54-881b-20d928cb8eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8054646d50741489dc193639b4c80e9, user_id=857e664f7fd94ffea53cd0d07dba2c4d, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.895323 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.896718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca49a0-1f22-4e54-881b-20d928cb8eec admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.897410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.898177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:15.900110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:15.913132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:15.933795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 910/1834] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:15 2022] PUT /identity/v3/projects/d8054646d50741489dc193639b4c80e9/users/857e664f7fd94ffea53cd0d07dba2c4d/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:15.940343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ca9f1-bda7-4290-8ded-4d1d72298d5c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:15.941148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ca9f1-bda7-4290-8ded-4d1d72298d5c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:15.941148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ca9f1-bda7-4290-8ded-4d1d72298d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:15.974721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-618ca9f1-bda7-4290-8ded-4d1d72298d5c None None] MFA Rules not processed for user `857e664f7fd94ffea53cd0d07dba2c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:15.976007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-618ca9f1-bda7-4290-8ded-4d1d72298d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:16.058856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 925/1835] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:15 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:16.060015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.064404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 911/1836] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:16.071178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ca9f1-bda7-4290-8ded-4d1d72298d5c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.128932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ca9f1-bda7-4290-8ded-4d1d72298d5c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.137145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769526c7-04c9-4c75-bfad-603a0be0fcff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.142397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.143056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.143308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.143586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.144211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.146176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.179530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.180143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.180430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.180702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.182449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:16.234764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 926/1837] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:16 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:16.235696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.245433 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc1a9e-cd74-4b1a-b91f-1ed04d52a7c0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.251054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.251849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.252230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.252581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.253203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.253298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.254696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.254868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.265037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 912/1838] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:16.265314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 927/1839] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:16.266583 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.269915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:16.271837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.281201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124e105-2854-43f8-bb61-5b4275138013 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.282077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351897cd-9737-4577-b96a-b721bcaba907 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.285034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.285676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.286073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.286322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.287809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.288539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.288890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.289235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] PATH_INFO: `/v3/projects/669a03a6f45f48ff8ec13e1dd11261a0/users/2f063621c175496183c948440a2c801c/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.307551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=669a03a6f45f48ff8ec13e1dd11261a0, user_id=2f063621c175496183c948440a2c801c, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.310767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.354844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.356275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.366144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 913/1840] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:16.375564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 928/1841] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:16 2022] PUT /identity/v3/projects/669a03a6f45f48ff8ec13e1dd11261a0/users/2f063621c175496183c948440a2c801c/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:16.380814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.394393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839e5c-0af6-4bfa-832e-532e86ccd5d3 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.478466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.523404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.524027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.524350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.524627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.525314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.527789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.546285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 914/1842] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:16.547515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.560772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50049ca3-de5d-4b2c-b8b1-ae59098f63e2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.566732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3b9d-a633-4f2e-ac5d-d7cc9796a934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.567901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3b9d-a633-4f2e-ac5d-d7cc9796a934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.569547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3b9d-a633-4f2e-ac5d-d7cc9796a934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.569872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3b9d-a633-4f2e-ac5d-d7cc9796a934 admin admin] PATH_INFO: `/v3/projects/669a03a6f45f48ff8ec13e1dd11261a0/users/2f063621c175496183c948440a2c801c/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.572311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c3b9d-a633-4f2e-ac5d-d7cc9796a934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=669a03a6f45f48ff8ec13e1dd11261a0, user_id=2f063621c175496183c948440a2c801c, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.575716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c3b9d-a633-4f2e-ac5d-d7cc9796a934 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.576550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d398873-7d36-4bdd-a291-848a82f20b33 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.582436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f728e4be7dbc43779d75e9df5f504d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effcdf1a12fd462f986d47992eb60083', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.590648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.603772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.604931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:16.605779 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.606190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 915/1843] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:16 2022] PUT /identity/v3/projects/669a03a6f45f48ff8ec13e1dd11261a0/users/2f063621c175496183c948440a2c801c/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:16.607435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:16.609454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.610044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3eb5f-f1d8-49c9-81e9-a55c69becc17 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.610401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3eb5f-f1d8-49c9-81e9-a55c69becc17 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.610688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3eb5f-f1d8-49c9-81e9-a55c69becc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.612573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.629487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 929/1844] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:14:16.633494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.651322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c2f3eb5f-f1d8-49c9-81e9-a55c69becc17 None None] MFA Rules not processed for user `2f063621c175496183c948440a2c801c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:16.651794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c2f3eb5f-f1d8-49c9-81e9-a55c69becc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:16.721097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c0d9f-d496-4e28-8856-af518e01b140 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.743224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 916/1845] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:16.743224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3eb5f-f1d8-49c9-81e9-a55c69becc17 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.788995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.790761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.791495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.791774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.822654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3eb5f-f1d8-49c9-81e9-a55c69becc17 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.828335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f728e4be7dbc43779d75e9df5f504d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effcdf1a12fd462f986d47992eb60083', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.829109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.829799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.830427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.831826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.833750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.857847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.859923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.871441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 930/1846] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:16.872908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.881448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174042-cf85-4ef4-8e0e-235a6aaf6d61 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:16.899533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:16.900277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:16.900507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:16.900924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:16.914539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:16.916118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:16.928482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 931/1847] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:16.971858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 917/1848] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:16 2022] POST /identity/v3/projects => generated 354 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:16.975879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:16.988953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b1f0-2882-45af-ae73-27641491ea98 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:17.048746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:17.092749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f728e4be7dbc43779d75e9df5f504d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effcdf1a12fd462f986d47992eb60083', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:17.094612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:17.095156 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:17.095585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:17.096423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:17.097381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:17.099859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:17.120624 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 932/1849] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:17.166598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb36d4f-f3b9-4c61-92d2-189ed93dc5b9 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:17.172921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:17.179832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:17.179832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:17.180035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] PATH_INFO: `/v3/users/3d98974d9aca46e09afa2a6b163c311f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:17.181733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d98974d9aca46e09afa2a6b163c311f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:17.183665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:17.312026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:17.323248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088042f1-9b5b-44b8-8325-6f4e189c9e04 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:17.335553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f728e4be7dbc43779d75e9df5f504d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effcdf1a12fd462f986d47992eb60083', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:17.336343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:17.336942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:17.337335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] PATH_INFO: `/v3/projects/2d20fde0d1c84ee68e5305abeb4168a5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:17.346374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2d20fde0d1c84ee68e5305abeb4168a5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:17.348336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:17.414988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d98974d9aca46e09afa2a6b163c311f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.420402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d98974d9aca46e09afa2a6b163c311f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.490630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d20fde0d1c84ee68e5305abeb4168a5', 'request_id': 'req-e2798bc1-59c4-4153-b92e-6034b604f860'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.496928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d20fde0d1c84ee68e5305abeb4168a5', 'request_id': 'req-e2798bc1-59c4-4153-b92e-6034b604f860'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.505181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d20fde0d1c84ee68e5305abeb4168a5', 'request_id': 'req-e2798bc1-59c4-4153-b92e-6034b604f860'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.531395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d98974d9aca46e09afa2a6b163c311f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.532371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d98974d9aca46e09afa2a6b163c311f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.568056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e71aecc-515f-48b1-8965-6726685a2c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d98974d9aca46e09afa2a6b163c311f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.589111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 918/1850] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:17 2022] DELETE /identity/v3/users/3d98974d9aca46e09afa2a6b163c311f => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:17.593989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:17.593989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:17.593989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:17.615136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:17.616384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:17.623776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:17.631117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] The token cache is being invalidate because project 2d20fde0d1c84ee68e5305abeb4168a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:17.631117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.631117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:17.637165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 933/1851] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:17 2022] DELETE /identity/v3/projects/2d20fde0d1c84ee68e5305abeb4168a5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:17.709738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 919/1852] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:17.721193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:17.723134 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:17.788982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2798bc1-59c4-4153-b92e-6034b604f860 tempest-VolumeQuotaClassesTest-1023238512 tempest-VolumeQuotaClassesTest-1023238512-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:17.852291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:17.852937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:17.853221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:17.853504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:17.865546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:17.867538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:17.881388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 934/1853] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:18.087870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:18.362873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:18.364739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:18.380890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7f8c5-3bac-4c13-bb42-d325a094d851 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:18.386001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:18.386647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:18.387194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:18.387469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] PATH_INFO: `/v3/users/decf9291a7094a0b8d7f78f81a2ced3d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:18.388928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=decf9291a7094a0b8d7f78f81a2ced3d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:18.391760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:18.535648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:18.537065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:18.560802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ceeadc-9d7c-4158-8720-99f2da312bbc service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:18.567966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:18.568805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:18.569607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:18.570302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] PATH_INFO: `/v3/projects/7a717494cd4a4526aace4b5563254943` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:18.571693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a717494cd4a4526aace4b5563254943)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:18.574249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:18.602403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'decf9291a7094a0b8d7f78f81a2ced3d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.603256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'decf9291a7094a0b8d7f78f81a2ced3d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.675224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'decf9291a7094a0b8d7f78f81a2ced3d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.675905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'decf9291a7094a0b8d7f78f81a2ced3d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.691264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-afc28643-71f6-4693-8c6a-c78977ffc167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'decf9291a7094a0b8d7f78f81a2ced3d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.693531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a717494cd4a4526aace4b5563254943', 'request_id': 'req-d1823457-ba5d-4825-bc86-415c7ded1f23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.699277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a717494cd4a4526aace4b5563254943', 'request_id': 'req-d1823457-ba5d-4825-bc86-415c7ded1f23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.704108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 920/1854] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:18 2022] DELETE /identity/v3/users/decf9291a7094a0b8d7f78f81a2ced3d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:18.709408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a717494cd4a4526aace4b5563254943', 'request_id': 'req-d1823457-ba5d-4825-bc86-415c7ded1f23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.709525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:18.709525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:18.709525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:18.727607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:18.728816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:18.786499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 921/1855] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:18.796218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:18.797740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:18.814804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] The token cache is being invalidate because project 7a717494cd4a4526aace4b5563254943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:18.815383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.817037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:18.819539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 935/1856] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:18 2022] DELETE /identity/v3/projects/7a717494cd4a4526aace4b5563254943 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:18.827050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:18.829234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:18.857078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4073cb-674f-4f5b-907e-9c1d48e730ef None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:18.861303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:18.861959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:18.862243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:18.862520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:18.865040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:18.866179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:18.879182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:18.880608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:18.891498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 922/1857] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:18.901840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823457-ba5d-4825-bc86-415c7ded1f23 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:18.906971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:18.907618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:18.908047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:18.908175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] PATH_INFO: `/v3/users/2f063621c175496183c948440a2c801c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:18.909705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f063621c175496183c948440a2c801c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:18.911436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:19.104480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f063621c175496183c948440a2c801c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.104480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f063621c175496183c948440a2c801c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.114837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f063621c175496183c948440a2c801c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.185148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f063621c175496183c948440a2c801c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.188426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f063621c175496183c948440a2c801c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.208334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 936/1858] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:18 2022] DELETE /identity/v3/users/2f063621c175496183c948440a2c801c => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:19.358001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:19.359471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:19.371634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b17e8-fbff-49fa-a0df-7b206a52d908 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:19.374351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:19.446752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:19.447398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:19.447681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:19.447959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] PATH_INFO: `/v3/projects/c08d9900ee684aa28ecad428bc9294f5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:19.449092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c08d9900ee684aa28ecad428bc9294f5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:19.451033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:19.573111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c08d9900ee684aa28ecad428bc9294f5', 'request_id': 'req-8fc9a299-27ad-4b28-9508-a89b1d65a155'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.689126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c08d9900ee684aa28ecad428bc9294f5', 'request_id': 'req-8fc9a299-27ad-4b28-9508-a89b1d65a155'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.701723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c08d9900ee684aa28ecad428bc9294f5', 'request_id': 'req-8fc9a299-27ad-4b28-9508-a89b1d65a155'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.717103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] The token cache is being invalidate because project c08d9900ee684aa28ecad428bc9294f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:19.717769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.722259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:19.738060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 923/1859] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:19 2022] DELETE /identity/v3/projects/c08d9900ee684aa28ecad428bc9294f5 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:19.742118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:19.744065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:19.760592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bad85-e3e4-4837-8e57-a8a8aa4c64d5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:19.850403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:19.851969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:19.884330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:19.885028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:19.885313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:19.888100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] PATH_INFO: `/v3/users/f728e4be7dbc43779d75e9df5f504d93` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:19.890787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f728e4be7dbc43779d75e9df5f504d93)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:19.892942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:19.961904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9a299-27ad-4b28-9508-a89b1d65a155 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:19.970134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:19.970512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:19.970673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:19.971260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] PATH_INFO: `/v3/projects/669a03a6f45f48ff8ec13e1dd11261a0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:19.973473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669a03a6f45f48ff8ec13e1dd11261a0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:19.974328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:20.069328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f728e4be7dbc43779d75e9df5f504d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.070650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f728e4be7dbc43779d75e9df5f504d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.074727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669a03a6f45f48ff8ec13e1dd11261a0', 'request_id': 'req-904ad641-7f57-43fc-8035-3826fc2fb4a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.085429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f728e4be7dbc43779d75e9df5f504d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.182131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669a03a6f45f48ff8ec13e1dd11261a0', 'request_id': 'req-904ad641-7f57-43fc-8035-3826fc2fb4a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.189081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669a03a6f45f48ff8ec13e1dd11261a0', 'request_id': 'req-904ad641-7f57-43fc-8035-3826fc2fb4a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.193993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:20.200193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] The token cache is being invalidate because project 669a03a6f45f48ff8ec13e1dd11261a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:20.200486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.201123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-904ad641-7f57-43fc-8035-3826fc2fb4a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.202640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 924/1860] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:19 2022] DELETE /identity/v3/projects/669a03a6f45f48ff8ec13e1dd11261a0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:20.207713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c620a-127d-4789-bfd7-f0626686ed0d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.208003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c620a-127d-4789-bfd7-f0626686ed0d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.208275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c620a-127d-4789-bfd7-f0626686ed0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.227192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fb4c620a-127d-4789-bfd7-f0626686ed0d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:20.228385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fb4c620a-127d-4789-bfd7-f0626686ed0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:20.264465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f728e4be7dbc43779d75e9df5f504d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.265085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d19d757b-c31c-4162-9ccc-70fd6c6d4727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f728e4be7dbc43779d75e9df5f504d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:20.278444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 937/1861] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:19 2022] DELETE /identity/v3/users/f728e4be7dbc43779d75e9df5f504d93 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:20.300528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 925/1862] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:20.302687 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660fcc3-5d48-4197-8a35-35a848a86c32 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.302979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660fcc3-5d48-4197-8a35-35a848a86c32 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.303254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660fcc3-5d48-4197-8a35-35a848a86c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.325053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4660fcc3-5d48-4197-8a35-35a848a86c32 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:20.326380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4660fcc3-5d48-4197-8a35-35a848a86c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:20.379943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:20.399584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 938/1863] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:20.404090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c620a-127d-4789-bfd7-f0626686ed0d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:20.414406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c620a-127d-4789-bfd7-f0626686ed0d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:20.419590 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:20.420226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.420478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.420756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.421410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:20.422332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:20.423927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:20.439478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 926/1864] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:20.440437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660fcc3-5d48-4197-8a35-35a848a86c32 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:20.450109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660fcc3-5d48-4197-8a35-35a848a86c32 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:20.455564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:20.456173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.456492 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.456717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.457357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:20.459297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:20.574057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 939/1865] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:20 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:20.574849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:20.586745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421356d-cae8-4384-b305-c7dd4aaacbba admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:20.649944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:20.650567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.650901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.651188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.652223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:20.654652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:20.764124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:20.766204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:20.770026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:20.775122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 927/1866] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:20 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:20.776376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:20.787516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bc922-cadf-4692-ba98-acc3131093dc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:20.793753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:20.795448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.795448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.795448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] PATH_INFO: `/v3/projects/effcdf1a12fd462f986d47992eb60083` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.796585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=effcdf1a12fd462f986d47992eb60083)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:20.799279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:20.808114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd60bc-7414-4038-8d55-41b6f821fc90 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:20.825900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:20.828567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.828567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.828567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.828567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:20.829831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:20.841611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 928/1867] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:20.846492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:20.856863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c309ca-6c0e-4e48-8598-0b3c21aa7d94 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:20.861926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:20.862605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.862863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.863135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.863810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:20.865378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:20.877598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 929/1868] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:20.880889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:20.891782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe05e8-932c-4b98-806b-f67533d5be9f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:20.896388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138560e4-a458-45fa-8241-6c8703f9356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:20.897067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138560e4-a458-45fa-8241-6c8703f9356a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:20.897359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138560e4-a458-45fa-8241-6c8703f9356a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:20.897658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138560e4-a458-45fa-8241-6c8703f9356a admin admin] PATH_INFO: `/v3/projects/05d93673dd314bdeb40aad53a07cdfd2/users/c2f474e8c056427199cd0f72bc590804/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:20.914285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138560e4-a458-45fa-8241-6c8703f9356a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d93673dd314bdeb40aad53a07cdfd2, user_id=c2f474e8c056427199cd0f72bc590804, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:20.917122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138560e4-a458-45fa-8241-6c8703f9356a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:21.043487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 930/1869] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:20 2022] PUT /identity/v3/projects/05d93673dd314bdeb40aad53a07cdfd2/users/c2f474e8c056427199cd0f72bc590804/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:21.043831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'effcdf1a12fd462f986d47992eb60083', 'request_id': 'req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:21.045309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:21.046183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:21.046469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:21.063204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'effcdf1a12fd462f986d47992eb60083', 'request_id': 'req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:21.084520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'effcdf1a12fd462f986d47992eb60083', 'request_id': 'req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:21.133413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:21.142342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] MFA Rules not processed for user `c2f474e8c056427199cd0f72bc590804`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:21.143815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:21.198649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] The token cache is being invalidate because project effcdf1a12fd462f986d47992eb60083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:21.199123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:21.200683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fc3f26d-ae04-4aef-960c-e118cd67f5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:21.203426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 940/1870] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:20 2022] DELETE /identity/v3/projects/effcdf1a12fd462f986d47992eb60083 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:21.208897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afcbc2-c94c-4d6f-b13e-ca07583ebf7a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:21.209312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afcbc2-c94c-4d6f-b13e-ca07583ebf7a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:21.209749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afcbc2-c94c-4d6f-b13e-ca07583ebf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:21.251064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 931/1871] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:21.252386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:21.253426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:21.258448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:21.258584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c1afcbc2-c94c-4d6f-b13e-ca07583ebf7a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:21.258584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c1afcbc2-c94c-4d6f-b13e-ca07583ebf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:21.355039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37033615-e596-4c15-ae18-9e226a23b77d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:21.361899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:21.362602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:21.363049 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:21.363432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:21.364197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:21.367738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:21.370181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 941/1872] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:21.371585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932094de-5c19-4266-875d-5bed4b13d08f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:21.371940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932094de-5c19-4266-875d-5bed4b13d08f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:21.372219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932094de-5c19-4266-875d-5bed4b13d08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:21.407375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-932094de-5c19-4266-875d-5bed4b13d08f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:21.410650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-932094de-5c19-4266-875d-5bed4b13d08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:21.457921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 942/1873] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:21.462916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932094de-5c19-4266-875d-5bed4b13d08f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:21.479222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932094de-5c19-4266-875d-5bed4b13d08f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:21.484993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:21.485796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:21.486196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:21.486545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:21.487347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:21.487790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:21.489193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 932/1874] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:21 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:21.489895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:21.495253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:21.521740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 943/1875] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:14:21.527528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:21.551433 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cf342-8fc7-4233-af62-6d8b04f91ee7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:21.629190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340fa7b-5064-42a2-b7e6-ba839f4c6195 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:21.638175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:21.639967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:21.641281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:21.641934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:21.642915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:21.645038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:21.646899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:21.649950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:21.649950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:21.649950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:21.649950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:21.651112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:22.212506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 933/1876] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:21 2022] POST /identity/v3/users => generated 298 bytes in 711 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:22.215564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:22.226696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87928d-69b1-4337-a433-6fb4f5323241 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:22.232073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:22.232791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.233149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.233497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.235060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:22.236572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:22.250549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 934/1877] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:22.266011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:22.287293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096d2a0-1aa8-4a85-ac6b-3e600d39b924 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:22.297671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:22.298557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.298921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.299203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] PATH_INFO: `/v3/projects/daf3967b72804187aa7a60f83bf73cd6/users/f340303b600340e780c371317297b558/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.333199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf3967b72804187aa7a60f83bf73cd6, user_id=f340303b600340e780c371317297b558, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:22.336075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:22.479258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 944/1878] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:21 2022] POST /identity/v3/projects => generated 387 bytes in 952 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:22.482504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:22.575875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347038d1-2dda-4f9b-ad27-24c2a7b2e445 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:22.585597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:22.594386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.594674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.595105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.596376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:22.597768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:22.642229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 935/1879] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:22 2022] PUT /identity/v3/projects/daf3967b72804187aa7a60f83bf73cd6/users/f340303b600340e780c371317297b558/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:22.645883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:22.710246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df118647-60e9-4235-8e5b-5f5623147910 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:22.714970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:22.715594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.715878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.716154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.716803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:22.718485 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:22.727464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 945/1880] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:22 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:22.733615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 936/1881] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:22.733615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:22.735461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:22.745247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c37d9-49a5-42fd-9f1e-a85f5924384a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:22.745849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25a986-d467-4b7e-91ec-681ac1e257e2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:22.751754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:22.752499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.752864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.753253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.754237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:22.758236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:22.783394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a6f7a-9c7a-4a51-b089-2959f437ea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:22.784424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a6f7a-9c7a-4a51-b089-2959f437ea8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.788315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a6f7a-9c7a-4a51-b089-2959f437ea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.788315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a6f7a-9c7a-4a51-b089-2959f437ea8e admin admin] PATH_INFO: `/v3/projects/daf3967b72804187aa7a60f83bf73cd6/users/f340303b600340e780c371317297b558/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.788515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 937/1882] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:22.789898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:22.791792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a6f7a-9c7a-4a51-b089-2959f437ea8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf3967b72804187aa7a60f83bf73cd6, user_id=f340303b600340e780c371317297b558, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:22.795149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a6f7a-9c7a-4a51-b089-2959f437ea8e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:22.820073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef558470-a7d2-4283-9acf-31acebf811ef admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:22.834546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 946/1883] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:22 2022] PUT /identity/v3/projects/daf3967b72804187aa7a60f83bf73cd6/users/f340303b600340e780c371317297b558/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:22.836034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.836358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.836703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.877867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] MFA Rules not processed for user `f340303b600340e780c371317297b558`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:22.879194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:22.968745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:22.972884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:22.973527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:22.975508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:22.978520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:22.980441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.003884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 938/1884] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:23.009837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.025776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b2554-23e5-4850-87fa-3e375d57c720 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:23.032976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee761f0-5fed-4eec-a244-fdc8cc5a6889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:23.033863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee761f0-5fed-4eec-a244-fdc8cc5a6889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.034358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee761f0-5fed-4eec-a244-fdc8cc5a6889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.035173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee761f0-5fed-4eec-a244-fdc8cc5a6889 admin admin] PATH_INFO: `/v3/projects/bda111f1f77246158a3164541a48d1e4/users/fb208606c598418a9866f6e5c2d3d812/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.049961 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 947/1885] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:23.064328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.066580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:23.095857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee761f0-5fed-4eec-a244-fdc8cc5a6889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda111f1f77246158a3164541a48d1e4, user_id=fb208606c598418a9866f6e5c2d3d812, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:23.101712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee761f0-5fed-4eec-a244-fdc8cc5a6889 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.154360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a77c-0835-449a-a8e1-b72434148a18 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:23.160393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ae961-0f96-4851-8302-078befae0edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:23.161025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ae961-0f96-4851-8302-078befae0edf service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.161308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ae961-0f96-4851-8302-078befae0edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.161887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ae961-0f96-4851-8302-078befae0edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.175846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ae961-0f96-4851-8302-078befae0edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:23.177473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ae961-0f96-4851-8302-078befae0edf service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.189980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 948/1886] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:23.221869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 939/1887] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:23 2022] PUT /identity/v3/projects/bda111f1f77246158a3164541a48d1e4/users/fb208606c598418a9866f6e5c2d3d812/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:23.223695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46443d-e6ab-4233-b4bf-bbebc0a9e54c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.223974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46443d-e6ab-4233-b4bf-bbebc0a9e54c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.224246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46443d-e6ab-4233-b4bf-bbebc0a9e54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.285423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fb46443d-e6ab-4233-b4bf-bbebc0a9e54c None None] MFA Rules not processed for user `fb208606c598418a9866f6e5c2d3d812`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:23.286738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fb46443d-e6ab-4233-b4bf-bbebc0a9e54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:23.342299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578c3a2-250c-4712-a519-1d3a28311570 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.343027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578c3a2-250c-4712-a519-1d3a28311570 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.344438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578c3a2-250c-4712-a519-1d3a28311570 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.346742 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 940/1888] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:14:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:14:23.352725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578c3a2-250c-4712-a519-1d3a28311570 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.421008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 949/1889] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:23.427276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd02bc0-33c7-4843-b69c-18ce0a3f5505 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.427459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd02bc0-33c7-4843-b69c-18ce0a3f5505 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.427851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd02bc0-33c7-4843-b69c-18ce0a3f5505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.452294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4dd02bc0-33c7-4843-b69c-18ce0a3f5505 None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:23.454116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4dd02bc0-33c7-4843-b69c-18ce0a3f5505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:23.493714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578c3a2-250c-4712-a519-1d3a28311570 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:23.495193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:23.495808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.496091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.496364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.512506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:23.538780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 950/1890] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:14:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:23.540247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd02bc0-33c7-4843-b69c-18ce0a3f5505 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.577457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:23.579753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.589844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 941/1891] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:23.590768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.592222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:23.618336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911a128-55ca-457b-9046-8c41b108eadc service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:23.619805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd02bc0-33c7-4843-b69c-18ce0a3f5505 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:23.624888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:23.625064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:23.625840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.626229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.626343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.626564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.626674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.626832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.627413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:23.629776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.637278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:23.651348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.662967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 942/1892] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:23.743708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 951/1893] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:23 2022] POST /identity/v3/projects => generated 387 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:23.747996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.759256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394b63-769b-4227-939e-5426de4d0ee1 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:23.815787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:23.816391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.816671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.816943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.817601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:23.819502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.922072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 943/1894] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:23 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:23.926129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.936780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449cd64-5209-41bd-acea-dbe3bf924f64 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:23.942137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:23.946297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:23.946297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:23.946297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:23.946297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:23.946297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:23.970561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 952/1895] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:23.974532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:23.985657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3c83d-9969-4fc3-892b-b26dd88a70bf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.002052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041aeb5-7e3c-4549-b5e1-467d88df1dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.002052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041aeb5-7e3c-4549-b5e1-467d88df1dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.002052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041aeb5-7e3c-4549-b5e1-467d88df1dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.002052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041aeb5-7e3c-4549-b5e1-467d88df1dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.002388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041aeb5-7e3c-4549-b5e1-467d88df1dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.002388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041aeb5-7e3c-4549-b5e1-467d88df1dcc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.010303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 944/1896] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:24.014948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:24.035984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f30d98-ed53-4b0c-837b-18d56bd77769 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.042067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.042732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.043016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.043295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] PATH_INFO: `/v3/projects/ada222de75bc4bb4b07509758fb4d3bb/users/d03ccd6cebbe45b6927db7c74aaebeff/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.058716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada222de75bc4bb4b07509758fb4d3bb, user_id=d03ccd6cebbe45b6927db7c74aaebeff, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.061461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.133363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 953/1897] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:24 2022] PUT /identity/v3/projects/ada222de75bc4bb4b07509758fb4d3bb/users/d03ccd6cebbe45b6927db7c74aaebeff/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:24.145902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b63fa-1984-4445-854d-1ca20c087266 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.146265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b63fa-1984-4445-854d-1ca20c087266 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.146553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b63fa-1984-4445-854d-1ca20c087266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.174708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b81b63fa-1984-4445-854d-1ca20c087266 None None] MFA Rules not processed for user `d03ccd6cebbe45b6927db7c74aaebeff`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:24.175941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b81b63fa-1984-4445-854d-1ca20c087266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:24.252254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 945/1898] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:24.255481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:24.324363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddb855-6b0f-4a39-8314-85a1e9a02bb3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.330519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.331173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.331459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.331736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.332410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.334396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.418923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:24.456446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 954/1899] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:24 2022] POST /identity/v3/projects => generated 387 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:24.460649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b63fa-1984-4445-854d-1ca20c087266 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:24.479896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b63fa-1984-4445-854d-1ca20c087266 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.556088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.556620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.556901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.557176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.557862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.559781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.675061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 946/1900] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:24 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:24.676293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:24.688124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f192db-b50b-4fe0-b991-7e4e7ad86067 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.698264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.698894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.699186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.699450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.700131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.701822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.732594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 955/1901] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:24.739091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:24.753482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d83be5-87cb-48ca-9487-7925ffdd7ac7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.759403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.760358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.760531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.760921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] PATH_INFO: `/v3/projects/3c2e368762764bbb8a0d62f7e34bc8e6/users/cbd61ac150374c8f9e1465d56734d4ed/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.792687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2e368762764bbb8a0d62f7e34bc8e6, user_id=cbd61ac150374c8f9e1465d56734d4ed, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.799396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.873818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 947/1902] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:24 2022] PUT /identity/v3/projects/3c2e368762764bbb8a0d62f7e34bc8e6/users/cbd61ac150374c8f9e1465d56734d4ed/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:24.878677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:24.889067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685232c9-1190-4148-bbb8-c144a482e741 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.922613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:24.947068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfddba-7603-41bc-9466-cc49e00f3a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.947679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfddba-7603-41bc-9466-cc49e00f3a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.947959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfddba-7603-41bc-9466-cc49e00f3a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.948234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfddba-7603-41bc-9466-cc49e00f3a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.948920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfddba-7603-41bc-9466-cc49e00f3a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.950843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfddba-7603-41bc-9466-cc49e00f3a2c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.961178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 956/1903] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:24.965353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:24.976134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cabfb-6b6b-4518-87fd-760899c07fec admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:24.980892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:24.981545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:24.981856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:24.982133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] PATH_INFO: `/v3/projects/3c2e368762764bbb8a0d62f7e34bc8e6/users/cbd61ac150374c8f9e1465d56734d4ed/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:24.984582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2e368762764bbb8a0d62f7e34bc8e6, user_id=cbd61ac150374c8f9e1465d56734d4ed, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:24.987242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:24.997198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 948/1904] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:24 2022] PUT /identity/v3/projects/3c2e368762764bbb8a0d62f7e34bc8e6/users/cbd61ac150374c8f9e1465d56734d4ed/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:25.008849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:25.008849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:25.009109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:25.037330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] MFA Rules not processed for user `cbd61ac150374c8f9e1465d56734d4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:25.038573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:25.085440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:25.110474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 957/1905] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:25.122780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:25.179115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fd619-e522-4b9f-b037-9d2be649ff97 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:25.182745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:25.183359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:25.183646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:25.183923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:25.194457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:25.195908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:25.206624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 949/1906] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:25.262852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:25.303273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:25.550749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:25.607695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:25.908067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:25.950457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:26.229238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:26.229238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:26.241995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe8ca-ff8c-44f1-b2cf-5def69ef18b2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:26.245661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3fdc6-b69e-458e-b214-2cf11022c211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:26.246218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fdc6-b69e-458e-b214-2cf11022c211 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:26.246500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fdc6-b69e-458e-b214-2cf11022c211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:26.246776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fdc6-b69e-458e-b214-2cf11022c211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:26.277336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:26.308728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3fdc6-b69e-458e-b214-2cf11022c211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:26.308728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3fdc6-b69e-458e-b214-2cf11022c211 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:26.321713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 958/1907] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:26.372381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:26.533886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:26.816917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:27.054139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:27.133498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:31.104121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:31.209807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c395c-4546-4901-b14c-140bd36339a3 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:31.216622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:31.217159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:31.217462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:31.217800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] PATH_INFO: `/v3/users/c2f474e8c056427199cd0f72bc590804` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:31.219927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f474e8c056427199cd0f72bc590804)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:31.223657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:31.456287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:31.458446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f474e8c056427199cd0f72bc590804'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:31.459263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f474e8c056427199cd0f72bc590804'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:31.578594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f474e8c056427199cd0f72bc590804'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:31.579284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f474e8c056427199cd0f72bc590804'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:31.603088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f474e8c056427199cd0f72bc590804'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:31.636427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 950/1908] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:31 2022] DELETE /identity/v3/users/c2f474e8c056427199cd0f72bc590804 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:31.645353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:31.645925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:31.645925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:31.674514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:31.675732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:31.767419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 959/1909] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:31.778002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:31.779850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:31.860006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cb99f-d209-45c1-9cf3-51ee9a249934 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:31.864758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc291868-3960-48f7-945a-9bc8256a9f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:31.865666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc291868-3960-48f7-945a-9bc8256a9f70 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:31.866071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc291868-3960-48f7-945a-9bc8256a9f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:31.866422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc291868-3960-48f7-945a-9bc8256a9f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:31.879636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc291868-3960-48f7-945a-9bc8256a9f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:31.881446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc291868-3960-48f7-945a-9bc8256a9f70 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:31.896649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 951/1910] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:32.081073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:32.082504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:32.095616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30161447-8cef-474f-b778-cbddac3ba6a7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:32.102182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:32.102182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:32.102182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:32.102182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] PATH_INFO: `/v3/users/d8c25a0d720241a597b594f4de2f829a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:32.103104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c25a0d720241a597b594f4de2f829a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:32.104997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:32.265660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:32.283372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c25a0d720241a597b594f4de2f829a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.284938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c25a0d720241a597b594f4de2f829a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.303442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c25a0d720241a597b594f4de2f829a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.361824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c25a0d720241a597b594f4de2f829a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.362371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c25a0d720241a597b594f4de2f829a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.377583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 960/1911] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:32 2022] DELETE /identity/v3/users/d8c25a0d720241a597b594f4de2f829a => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:32.381509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:32.382035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:32.382158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:32.414051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:32.415325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:32.465467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:32.467996 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:32.508374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 952/1912] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:32.519781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:32.522118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:32.553886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9869e-a04b-4527-9e06-50395af64ab5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:32.559301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:32.559933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:32.560213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:32.560485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] PATH_INFO: `/v3/projects/05d93673dd314bdeb40aad53a07cdfd2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:32.561632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d93673dd314bdeb40aad53a07cdfd2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:32.564279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:32.592621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae798af6-a69c-4f50-9877-c40c96f5f015 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:32.596394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:32.597005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:32.597288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:32.597596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:32.607717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:32.609758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:32.620160 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 953/1913] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:32.702587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d93673dd314bdeb40aad53a07cdfd2', 'request_id': 'req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.709076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d93673dd314bdeb40aad53a07cdfd2', 'request_id': 'req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.715310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d93673dd314bdeb40aad53a07cdfd2', 'request_id': 'req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.826458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] The token cache is being invalidate because project 05d93673dd314bdeb40aad53a07cdfd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:32.826917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.827603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:32.832157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 961/1914] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:32 2022] DELETE /identity/v3/projects/05d93673dd314bdeb40aad53a07cdfd2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:32.833370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:32.842978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f7c55-c22d-427d-994c-c942bbedd919 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:32.849025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:32.915429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:32.916535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:32.916535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:32.916914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] PATH_INFO: `/v3/users/f340303b600340e780c371317297b558` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:32.918731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f340303b600340e780c371317297b558)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:32.920727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:33.102046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f340303b600340e780c371317297b558'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.102710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f340303b600340e780c371317297b558'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.180390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:33.181907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:33.185187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f340303b600340e780c371317297b558'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.186119 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f340303b600340e780c371317297b558'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.197276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b71cd3-22a1-48c7-b6ce-232ce4a570c4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:33.205547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:33.205547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:33.205547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:33.205547 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] PATH_INFO: `/v3/projects/660b5fc09131429e9fcb9d3486cbfe92` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:33.206256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f340303b600340e780c371317297b558'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.206344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=660b5fc09131429e9fcb9d3486cbfe92)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:33.207270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:33.218007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 954/1915] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:32 2022] DELETE /identity/v3/users/f340303b600340e780c371317297b558 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:33.333828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:33.346255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '660b5fc09131429e9fcb9d3486cbfe92', 'request_id': 'req-bd3b9333-a59c-4c97-a0a4-4e454d507a98'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.351815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '660b5fc09131429e9fcb9d3486cbfe92', 'request_id': 'req-bd3b9333-a59c-4c97-a0a4-4e454d507a98'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.358042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '660b5fc09131429e9fcb9d3486cbfe92', 'request_id': 'req-bd3b9333-a59c-4c97-a0a4-4e454d507a98'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.418477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] The token cache is being invalidate because project 660b5fc09131429e9fcb9d3486cbfe92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:33.418861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.419569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.424976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 962/1916] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:33 2022] DELETE /identity/v3/projects/660b5fc09131429e9fcb9d3486cbfe92 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:33.426175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:33.429069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:33.527713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7380a-d047-445b-be1e-a1105095648e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:33.595220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:33.596094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:33.596564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:33.596943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] PATH_INFO: `/v3/users/857e664f7fd94ffea53cd0d07dba2c4d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:33.612437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=857e664f7fd94ffea53cd0d07dba2c4d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:33.612437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:33.639256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:33.792129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '857e664f7fd94ffea53cd0d07dba2c4d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.792788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '857e664f7fd94ffea53cd0d07dba2c4d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.809457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:33.810922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:33.824497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9333-a59c-4c97-a0a4-4e454d507a98 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:33.829874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:33.830584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:33.830946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:33.831766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] PATH_INFO: `/v3/projects/daf3967b72804187aa7a60f83bf73cd6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:33.833814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf3967b72804187aa7a60f83bf73cd6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:33.835798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:33.867824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '857e664f7fd94ffea53cd0d07dba2c4d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.868493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '857e664f7fd94ffea53cd0d07dba2c4d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.881166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c77924b8-f059-4188-b5b1-0a8979b7e8cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '857e664f7fd94ffea53cd0d07dba2c4d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.894129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 955/1917] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:33 2022] DELETE /identity/v3/users/857e664f7fd94ffea53cd0d07dba2c4d => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:33.982550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf3967b72804187aa7a60f83bf73cd6', 'request_id': 'req-2cda0fe7-9517-4c03-89fa-6606646ccdc3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.988641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf3967b72804187aa7a60f83bf73cd6', 'request_id': 'req-2cda0fe7-9517-4c03-89fa-6606646ccdc3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:33.996561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf3967b72804187aa7a60f83bf73cd6', 'request_id': 'req-2cda0fe7-9517-4c03-89fa-6606646ccdc3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.087048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] The token cache is being invalidate because project daf3967b72804187aa7a60f83bf73cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:34.087346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.088054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2cda0fe7-9517-4c03-89fa-6606646ccdc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.095863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9966b-64be-4c91-842f-cf38f91cd6be None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.097332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9966b-64be-4c91-842f-cf38f91cd6be None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.097332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9966b-64be-4c91-842f-cf38f91cd6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.102804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 963/1918] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:33 2022] DELETE /identity/v3/projects/daf3967b72804187aa7a60f83bf73cd6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:34.122123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-22f9966b-64be-4c91-842f-cf38f91cd6be None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:34.125351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-22f9966b-64be-4c91-842f-cf38f91cd6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:34.216631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 956/1919] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:34.224760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffaec8-287b-459a-896f-9bcae1edfc2a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.225096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffaec8-287b-459a-896f-9bcae1edfc2a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.225381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffaec8-287b-459a-896f-9bcae1edfc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.248085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8cffaec8-287b-459a-896f-9bcae1edfc2a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:34.249310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8cffaec8-287b-459a-896f-9bcae1edfc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:34.258916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 964/1920] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:34.263512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9966b-64be-4c91-842f-cf38f91cd6be None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.274289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9966b-64be-4c91-842f-cf38f91cd6be None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.278805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.279411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.279686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.279958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.280593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:34.281051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.282624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.293989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 957/1921] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:34.302884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffaec8-287b-459a-896f-9bcae1edfc2a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.312708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffaec8-287b-459a-896f-9bcae1edfc2a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.317778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.318365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.318713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.318997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.319854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.321824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.362799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.364622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:34.378782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63608b-28ea-4908-9d70-534c776ca9b5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.383582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.387799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.387799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.387799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] PATH_INFO: `/v3/projects/d8054646d50741489dc193639b4c80e9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.387799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8054646d50741489dc193639b4c80e9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.389700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.438450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 965/1922] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/projects => generated 407 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:34.442454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.502402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:34.507264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ca19-0442-4015-b583-98f4045c05ad admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.511607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8054646d50741489dc193639b4c80e9', 'request_id': 'req-99b38b26-d615-49e5-8507-258c0f46b935'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.526086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:34.564391 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.564997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.565275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.565548 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.566228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.568173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.584358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8054646d50741489dc193639b4c80e9', 'request_id': 'req-99b38b26-d615-49e5-8507-258c0f46b935'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.590504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8054646d50741489dc193639b4c80e9', 'request_id': 'req-99b38b26-d615-49e5-8507-258c0f46b935'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.603004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] The token cache is being invalidate because project d8054646d50741489dc193639b4c80e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:34.603312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.603949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-99b38b26-d615-49e5-8507-258c0f46b935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:34.605498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 958/1923] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:34 2022] DELETE /identity/v3/projects/d8054646d50741489dc193639b4c80e9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:34.610873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf90a0-3769-4f82-8401-5dfc599b14e9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.611172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf90a0-3769-4f82-8401-5dfc599b14e9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.611436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf90a0-3769-4f82-8401-5dfc599b14e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.633635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-26cf90a0-3769-4f82-8401-5dfc599b14e9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:34.634895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-26cf90a0-3769-4f82-8401-5dfc599b14e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:34.644449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 959/1924] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:34.653874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69d77a-2e88-49a7-9e67-d7a95d5a653c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.654221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69d77a-2e88-49a7-9e67-d7a95d5a653c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.654597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69d77a-2e88-49a7-9e67-d7a95d5a653c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.663189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 966/1925] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/users => generated 304 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:34.678624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ac69d77a-2e88-49a7-9e67-d7a95d5a653c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:34.679925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ac69d77a-2e88-49a7-9e67-d7a95d5a653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:34.684579 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.686434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:34.688653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 960/1926] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:34.695515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69d77a-2e88-49a7-9e67-d7a95d5a653c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.698464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ab51d-8bb1-420b-9eb3-038a5a8e164c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.704436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.705051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.705337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.705666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.705774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69d77a-2e88-49a7-9e67-d7a95d5a653c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.706318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.707922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.711619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.712220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.715749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.715749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.715749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:34.715749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.715749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.717866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 967/1927] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:34.722466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.728219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 961/1928] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:14:34.732351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.736386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d14486-b7a2-46d9-8f7c-457755b733e5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.741049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.741675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.741962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.742238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.742611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e726c-3c7a-4ea9-ae91-6690b8c98b30 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.743003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.744568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.747456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.748062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.748375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.748653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.749390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.751581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.756084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 968/1929] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:34.760362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.771890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec95957-69a2-488f-8c56-9849dce3930a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:34.776911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2540b-f6fa-48c4-a539-da1da1294d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:34.777799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2540b-f6fa-48c4-a539-da1da1294d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.778481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2540b-f6fa-48c4-a539-da1da1294d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.779057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2540b-f6fa-48c4-a539-da1da1294d51 admin admin] PATH_INFO: `/v3/projects/30dfdaf04fae4c4dbd97043024d6d4be/users/8d2934973ef44a3492db375a8a5c4fa9/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.796579 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2540b-f6fa-48c4-a539-da1da1294d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30dfdaf04fae4c4dbd97043024d6d4be, user_id=8d2934973ef44a3492db375a8a5c4fa9, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:34.799484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2540b-f6fa-48c4-a539-da1da1294d51 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:34.866219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 962/1930] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:34.870503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:34.897601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 969/1931] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:34 2022] PUT /identity/v3/projects/30dfdaf04fae4c4dbd97043024d6d4be/users/8d2934973ef44a3492db375a8a5c4fa9/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:34.909742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2af7-60a6-4b75-ab25-e4789511f2ee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:34.910357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2af7-60a6-4b75-ab25-e4789511f2ee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:34.910959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2af7-60a6-4b75-ab25-e4789511f2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:34.940438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d02c2af7-60a6-4b75-ab25-e4789511f2ee None None] MFA Rules not processed for user `8d2934973ef44a3492db375a8a5c4fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:34.942052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d02c2af7-60a6-4b75-ab25-e4789511f2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:34.964831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e524-3070-425b-95ad-f7b8320eed10 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.026562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 970/1932] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:35.031318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2af7-60a6-4b75-ab25-e4789511f2ee None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.038565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.039488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.039947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.040426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.041239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.044615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.102968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:35.111257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2af7-60a6-4b75-ab25-e4789511f2ee None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.116602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.117325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.117741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.118229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.118969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.121654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.126624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 963/1933] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:34 2022] POST /identity/v3/users => generated 294 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:35.132012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.143930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36acc92-0568-4332-82bf-2882193fb21a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.152991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.157258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.157258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.157258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.157258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.157258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.167208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 964/1934] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:35.171015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.176646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:35.181771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017edc-ab4a-4c66-b70a-5227bc6e360b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.186552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.187182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.187442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.187721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.188336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.189922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.203195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 965/1935] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:35.207625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.220539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e833-8a41-4443-822a-aba11efbeae3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.229961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d3d4-1499-4ff5-9a82-741530ca993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.230650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d3d4-1499-4ff5-9a82-741530ca993a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.230944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d3d4-1499-4ff5-9a82-741530ca993a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.231635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d3d4-1499-4ff5-9a82-741530ca993a admin admin] PATH_INFO: `/v3/projects/2d860627c59f40a2bcc5390a88bb8733/users/c06652435efe4752b6fee46265fc8262/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.233639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 971/1936] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:35 2022] POST /identity/v3/projects => generated 407 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:35.236937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.266285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d3d4-1499-4ff5-9a82-741530ca993a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d860627c59f40a2bcc5390a88bb8733, user_id=c06652435efe4752b6fee46265fc8262, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.271067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d3d4-1499-4ff5-9a82-741530ca993a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.313689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b2b2-6d0c-4852-be76-7761d695b69a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.325318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.326062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.326356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.326643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.327479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.329701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.347405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 966/1937] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:35 2022] PUT /identity/v3/projects/2d860627c59f40a2bcc5390a88bb8733/users/c06652435efe4752b6fee46265fc8262/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:35.358035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e6134-39fd-4214-a4c0-cf9bec320ce2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.358577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e6134-39fd-4214-a4c0-cf9bec320ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.358577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e6134-39fd-4214-a4c0-cf9bec320ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.400373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-368e6134-39fd-4214-a4c0-cf9bec320ce2 None None] MFA Rules not processed for user `c06652435efe4752b6fee46265fc8262`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:35.401622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-368e6134-39fd-4214-a4c0-cf9bec320ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:35.447246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 972/1938] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:35 2022] POST /identity/v3/users => generated 310 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:35.449309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.478700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09eca00-bfa8-4563-bdaa-cbaff2d299bb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.491457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 967/1939] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:35.495893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e6134-39fd-4214-a4c0-cf9bec320ce2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.552361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.553607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.554054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.554337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.554998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.556625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.567703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 973/1940] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:35.571911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.578331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e6134-39fd-4214-a4c0-cf9bec320ce2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.584314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684ecc3-d664-46c5-a65c-394b17188ef2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.585817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.586663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.588998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.588998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.588998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.589820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.590567 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.590882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.590976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.591293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] PATH_INFO: `/v3/projects/f08f40f41d4c4612806feb672a9c4b55/users/a76db9c53ed545869cb3532868e165ee/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.608516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08f40f41d4c4612806feb672a9c4b55, user_id=a76db9c53ed545869cb3532868e165ee, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.612159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.746405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 974/1941] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:35 2022] PUT /identity/v3/projects/f08f40f41d4c4612806feb672a9c4b55/users/a76db9c53ed545869cb3532868e165ee/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:35.751540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 968/1942] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:35 2022] POST /identity/v3/projects => generated 387 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:35.751901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.753277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.815608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:35.838376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf399f26-c508-45d6-a02d-29ffdf0d7a78 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.842196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75da21d-604d-47da-91f1-41eece6dbe26 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.844097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.844797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.845118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.845344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.846059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.847979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.848290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.848968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.849278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.849551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.850304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.851968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.863643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 975/1943] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:35.864663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:35.874068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.885351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735fd2a-9c9e-4184-bf26-b1d0b8755380 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.895102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bb1a9-3583-4611-a551-2bdafecc573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:35.903179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb1a9-3583-4611-a551-2bdafecc573d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.903179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb1a9-3583-4611-a551-2bdafecc573d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.903179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb1a9-3583-4611-a551-2bdafecc573d admin admin] PATH_INFO: `/v3/projects/f08f40f41d4c4612806feb672a9c4b55/users/a76db9c53ed545869cb3532868e165ee/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.903179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bb1a9-3583-4611-a551-2bdafecc573d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08f40f41d4c4612806feb672a9c4b55, user_id=a76db9c53ed545869cb3532868e165ee, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:35.903179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bb1a9-3583-4611-a551-2bdafecc573d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:35.913128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 976/1944] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:35 2022] PUT /identity/v3/projects/f08f40f41d4c4612806feb672a9c4b55/users/a76db9c53ed545869cb3532868e165ee/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:35.931000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41402b4-00ec-4d24-8032-6b40374ae36a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:35.933608 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41402b4-00ec-4d24-8032-6b40374ae36a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:35.933762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 969/1945] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:35 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:35.933876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41402b4-00ec-4d24-8032-6b40374ae36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:35.935962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:35.950868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80399be8-1292-4269-8f33-42c17013992d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:35.964157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a41402b4-00ec-4d24-8032-6b40374ae36a None None] MFA Rules not processed for user `a76db9c53ed545869cb3532868e165ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:35.964774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a41402b4-00ec-4d24-8032-6b40374ae36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:36.039092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:36.040089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:36.040552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:36.040903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:36.042665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:36.045909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:36.064629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 970/1946] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:36.064629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:36.065347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:36.073499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 977/1947] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:36.074491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41402b4-00ec-4d24-8032-6b40374ae36a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:36.091631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41402b4-00ec-4d24-8032-6b40374ae36a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:36.098601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:36.099247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:36.099589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:36.099872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] PATH_INFO: `/v3/projects/4137246182ab41d693d21323aaecc5cf/users/d734741d7294486085210290d4affa61/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:36.119783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4137246182ab41d693d21323aaecc5cf, user_id=d734741d7294486085210290d4affa61, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:36.123775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:36.160455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde45e41-4cd5-4c79-8fde-ad8bbdc4d030 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:36.164604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:36.165278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:36.165581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:36.165896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:36.167916 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:36.191263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 978/1948] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:36 2022] PUT /identity/v3/projects/4137246182ab41d693d21323aaecc5cf/users/d734741d7294486085210290d4affa61/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:36.194098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:36.199411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:36.242642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:36.246027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:36.260257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 971/1949] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:36.261204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:36.268690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138d647-1a2a-4599-9cc6-5333b31924f8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:36.273852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:36.275097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:36.275349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120399-bf97-4306-965b-c1545f3d5289 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:36.275506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:36.275955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:36.338960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:36.346796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:36.371162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde8d9e-8a6c-46fe-91d1-4d973e53e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:36.371925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8d9e-8a6c-46fe-91d1-4d973e53e900 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:36.372225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8d9e-8a6c-46fe-91d1-4d973e53e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:36.372981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8d9e-8a6c-46fe-91d1-4d973e53e900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:36.373188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde8d9e-8a6c-46fe-91d1-4d973e53e900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:36.374578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:36.374896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde8d9e-8a6c-46fe-91d1-4d973e53e900 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:36.376053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:36.389432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 979/1951] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:36.389659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 972/1951] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:36.391157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:36.401808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38754d-b165-4af6-8c79-7857b3630af1 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:36.407820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:36.409333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:36.409553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:36.409928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] PATH_INFO: `/v3/projects/4137246182ab41d693d21323aaecc5cf/users/d734741d7294486085210290d4affa61/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:36.412341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4137246182ab41d693d21323aaecc5cf, user_id=d734741d7294486085210290d4affa61, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:36.415111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:36.426436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 980/1952] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:36 2022] PUT /identity/v3/projects/4137246182ab41d693d21323aaecc5cf/users/d734741d7294486085210290d4affa61/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:36.442717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3bb08-7448-4a1b-9949-9ffacdde4790 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:36.443208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3bb08-7448-4a1b-9949-9ffacdde4790 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:36.443564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3bb08-7448-4a1b-9949-9ffacdde4790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:36.482036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f5e3bb08-7448-4a1b-9949-9ffacdde4790 None None] MFA Rules not processed for user `d734741d7294486085210290d4affa61`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:36.483797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f5e3bb08-7448-4a1b-9949-9ffacdde4790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:36.567861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 973/1953] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:36.985282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:37.043393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf6075-c12b-432a-94b5-794f686f839a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:37.047297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:37.047907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:37.048191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:37.048488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:37.057662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:37.059111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:37.069059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 981/1954] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:37.187095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3bb08-7448-4a1b-9949-9ffacdde4790 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:37.197513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3bb08-7448-4a1b-9949-9ffacdde4790 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:37.201131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d44f5-7736-4ba6-a071-e0ba0799e651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:37.201752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d44f5-7736-4ba6-a071-e0ba0799e651 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:37.202037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d44f5-7736-4ba6-a071-e0ba0799e651 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:37.202341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d44f5-7736-4ba6-a071-e0ba0799e651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:37.260973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d44f5-7736-4ba6-a071-e0ba0799e651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:37.262735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d44f5-7736-4ba6-a071-e0ba0799e651 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:37.273301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 974/1955] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:37 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:39.786905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:39.855815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be60fa-dd5e-48c7-a1ba-76e07eec26df service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:39.861192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:39.861854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:39.862139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:39.863016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] PATH_INFO: `/v3/users/c06652435efe4752b6fee46265fc8262` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:39.864142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06652435efe4752b6fee46265fc8262)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:39.866245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:40.033648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06652435efe4752b6fee46265fc8262'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:40.034244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06652435efe4752b6fee46265fc8262'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:40.050895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06652435efe4752b6fee46265fc8262'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:40.053678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:40.130270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06652435efe4752b6fee46265fc8262'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:40.130957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06652435efe4752b6fee46265fc8262'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:40.150902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 982/1956] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:39 2022] DELETE /identity/v3/users/c06652435efe4752b6fee46265fc8262 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:40.155335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:40.155819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:40.156130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:40.180226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:40.194055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:40.242338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:40.269850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 975/1957] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:40.280826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:40.283235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:40.352259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a325d-b9db-456b-8546-9d10b7662000 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:40.356515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:40.357171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:40.357455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:40.361855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:40.373047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:40.374650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:40.392573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 983/1958] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:40.852692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:40.854183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:40.886898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e70f5-c3ac-4769-b8b4-093b80097c77 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:40.892274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:40.893517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:40.893925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:40.894461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] PATH_INFO: `/v3/projects/2d860627c59f40a2bcc5390a88bb8733` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:40.895727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d860627c59f40a2bcc5390a88bb8733)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:40.897757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:41.020425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d860627c59f40a2bcc5390a88bb8733', 'request_id': 'req-96393509-d58a-4177-8e65-99f6b401b816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.090021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d860627c59f40a2bcc5390a88bb8733', 'request_id': 'req-96393509-d58a-4177-8e65-99f6b401b816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.096465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d860627c59f40a2bcc5390a88bb8733', 'request_id': 'req-96393509-d58a-4177-8e65-99f6b401b816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.107675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] The token cache is being invalidate because project 2d860627c59f40a2bcc5390a88bb8733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:41.107969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.108615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.110228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 976/1959] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:40 2022] DELETE /identity/v3/projects/2d860627c59f40a2bcc5390a88bb8733 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:41.114409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:41.123945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d608-e806-4c45-ad11-656a6302a78c service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:41.180419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:41.181051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:41.181346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:41.181653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] PATH_INFO: `/v3/users/d734741d7294486085210290d4affa61` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:41.182832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d734741d7294486085210290d4affa61)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:41.184749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:41.397003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd734741d7294486085210290d4affa61'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.397685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd734741d7294486085210290d4affa61'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.410887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd734741d7294486085210290d4affa61'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.476639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:41.478097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:41.479170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd734741d7294486085210290d4affa61'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.479797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-482b3223-c9fa-4534-b254-45096036847e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd734741d7294486085210290d4affa61'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.490848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393509-d58a-4177-8e65-99f6b401b816 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:41.492952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 984/1960] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:41 2022] DELETE /identity/v3/users/d734741d7294486085210290d4affa61 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:41.495543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:41.496178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:41.496446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:41.496725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] PATH_INFO: `/v3/users/fb208606c598418a9866f6e5c2d3d812` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:41.497874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb208606c598418a9866f6e5c2d3d812)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:41.499898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:41.721351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb208606c598418a9866f6e5c2d3d812'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.722160 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb208606c598418a9866f6e5c2d3d812'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.782813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb208606c598418a9866f6e5c2d3d812'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.783440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb208606c598418a9866f6e5c2d3d812'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.796285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb208606c598418a9866f6e5c2d3d812'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:41.808962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 977/1961] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:41 2022] DELETE /identity/v3/users/fb208606c598418a9866f6e5c2d3d812 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:41.813899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c526b-af49-488d-a150-810d10c1f7da None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:41.814191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c526b-af49-488d-a150-810d10c1f7da None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:41.814470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c526b-af49-488d-a150-810d10c1f7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:41.827463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:41.835099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-144c526b-af49-488d-a150-810d10c1f7da None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:41.836313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-144c526b-af49-488d-a150-810d10c1f7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:41.897953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 985/1962] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:41.899770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:41.901068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:41.916261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c526b-af49-488d-a150-810d10c1f7da None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:41.919224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-144c526b-af49-488d-a150-810d10c1f7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:41.929200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058b713-f5e8-4bca-a215-8fdb10267f51 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:41.934120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:41.935001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:41.935354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:41.935741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] PATH_INFO: `/v3/projects/4137246182ab41d693d21323aaecc5cf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:41.937024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4137246182ab41d693d21323aaecc5cf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:41.939777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.000136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c526b-af49-488d-a150-810d10c1f7da None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.004523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377a19-3934-4cf4-bb09-d0900aab9f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.005142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377a19-3934-4cf4-bb09-d0900aab9f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.005422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377a19-3934-4cf4-bb09-d0900aab9f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.005733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377a19-3934-4cf4-bb09-d0900aab9f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.016302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03377a19-3934-4cf4-bb09-d0900aab9f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.017846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03377a19-3934-4cf4-bb09-d0900aab9f26 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.028645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 986/1963] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:42.032095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:42.060250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4137246182ab41d693d21323aaecc5cf', 'request_id': 'req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.110165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4137246182ab41d693d21323aaecc5cf', 'request_id': 'req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.118528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4137246182ab41d693d21323aaecc5cf', 'request_id': 'req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.130761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] The token cache is being invalidate because project 4137246182ab41d693d21323aaecc5cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:42.130974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.132267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e2ecc5fc-1f5c-4131-8b53-bf0b9f0abcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.134562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 978/1964] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:41 2022] DELETE /identity/v3/projects/4137246182ab41d693d21323aaecc5cf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:42.140901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f1575-9995-4e39-99d5-5d5923cfa143 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.141377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f1575-9995-4e39-99d5-5d5923cfa143 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.141505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f1575-9995-4e39-99d5-5d5923cfa143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.161525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-003f1575-9995-4e39-99d5-5d5923cfa143 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:42.162898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-003f1575-9995-4e39-99d5-5d5923cfa143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:42.230233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 987/1965] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:42.242137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c723df-7a7c-4cbb-8358-2daf35a0f3fe None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.242557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c723df-7a7c-4cbb-8358-2daf35a0f3fe None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.242706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c723df-7a7c-4cbb-8358-2daf35a0f3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.268037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-68c723df-7a7c-4cbb-8358-2daf35a0f3fe None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:42.269300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-68c723df-7a7c-4cbb-8358-2daf35a0f3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:42.277481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 979/1966] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:42.282067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f1575-9995-4e39-99d5-5d5923cfa143 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.292137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f1575-9995-4e39-99d5-5d5923cfa143 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.297524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.298172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.298461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.298739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.299392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:42.299733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.301431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.317484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 988/1967] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:42.319145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c723df-7a7c-4cbb-8358-2daf35a0f3fe None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.328753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c723df-7a7c-4cbb-8358-2daf35a0f3fe None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.333869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.334849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.334849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.342367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.342367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.342367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.429746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 980/1968] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:42 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:42.430795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.441176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8003d3-da7a-4932-962a-bb091dd2edca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.475933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:42.488514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.490295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:42.501208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.502049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.502457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.502738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.503377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.505591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.563797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40cde-f414-43b5-b508-4350b48f9114 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.569199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.569199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.569410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.569683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] PATH_INFO: `/v3/projects/bda111f1f77246158a3164541a48d1e4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.570817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda111f1f77246158a3164541a48d1e4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.572713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.624262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 989/1969] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:42 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:42.628252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.637896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77e096-51db-41e5-91ef-5a2923da47b3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.642486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.643058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.643342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.643610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.644235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.645847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.657265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 990/1970] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:42.660810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.670190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde86c3-cea2-435e-a3f9-1004657f4456 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.675423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.676002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.676285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.676556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.677184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.678756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.689104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 991/1971] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:42.693023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.694958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda111f1f77246158a3164541a48d1e4', 'request_id': 'req-78f48a78-125f-45c6-a3c6-d7d05013492e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.702493 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2b204-ce6b-482b-af0d-ff1fa4f12935 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.759648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88726b-0591-40ab-8257-2c0ae7cb38bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.760300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88726b-0591-40ab-8257-2c0ae7cb38bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.760578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88726b-0591-40ab-8257-2c0ae7cb38bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.760851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88726b-0591-40ab-8257-2c0ae7cb38bc admin admin] PATH_INFO: `/v3/projects/3ba3f2b6ecf540759b706043dde7c248/users/5ce568ea7abb4674ab38463a36c7c3e8/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.774824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88726b-0591-40ab-8257-2c0ae7cb38bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba3f2b6ecf540759b706043dde7c248, user_id=5ce568ea7abb4674ab38463a36c7c3e8, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.777481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88726b-0591-40ab-8257-2c0ae7cb38bc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.778233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda111f1f77246158a3164541a48d1e4', 'request_id': 'req-78f48a78-125f-45c6-a3c6-d7d05013492e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.785855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda111f1f77246158a3164541a48d1e4', 'request_id': 'req-78f48a78-125f-45c6-a3c6-d7d05013492e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.799751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] The token cache is being invalidate because project bda111f1f77246158a3164541a48d1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:42.800041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.800715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:42.802263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 981/1972] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:42 2022] DELETE /identity/v3/projects/bda111f1f77246158a3164541a48d1e4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:42.806102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.807404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:42.820006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f48a78-125f-45c6-a3c6-d7d05013492e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:42.824587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:42.825184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.826604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.826604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] PATH_INFO: `/v3/users/d03ccd6cebbe45b6927db7c74aaebeff` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.826947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03ccd6cebbe45b6927db7c74aaebeff)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:42.829348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:42.846269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:42.847835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 992/1973] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:42 2022] PUT /identity/v3/projects/3ba3f2b6ecf540759b706043dde7c248/users/5ce568ea7abb4674ab38463a36c7c3e8/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:42.858279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:42.858565 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:42.858864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:42.885970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] MFA Rules not processed for user `5ce568ea7abb4674ab38463a36c7c3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:42.887413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:42.969600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 993/1974] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:42.982152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:42.984984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:43.012934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:43.019567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03ccd6cebbe45b6927db7c74aaebeff'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:43.020881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03ccd6cebbe45b6927db7c74aaebeff'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:43.071763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fed546-219b-4197-85b5-4c3a17f22f7e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:43.075477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95204bd338a4a99a510a67c4ab97532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:43.076080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:43.076360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:43.076630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:43.078445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:43.087130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03ccd6cebbe45b6927db7c74aaebeff'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:43.087815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03ccd6cebbe45b6927db7c74aaebeff'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:43.089874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:43.091326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:43.100985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 994/1975] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:43.112952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03ccd6cebbe45b6927db7c74aaebeff'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:43.146606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 982/1976] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:42 2022] DELETE /identity/v3/users/d03ccd6cebbe45b6927db7c74aaebeff => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:43.712860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:43.714454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:43.795188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb6f48-e96f-491a-ac75-e45165d5636a service glance] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:43.851735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:43.852368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:43.852648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:43.852986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] PATH_INFO: `/v3/projects/ada222de75bc4bb4b07509758fb4d3bb` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:43.854064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada222de75bc4bb4b07509758fb4d3bb)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:43.855981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:43.939382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada222de75bc4bb4b07509758fb4d3bb', 'request_id': 'req-0d926c40-6711-4741-be41-92b323960af2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:43.940595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:43.945658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada222de75bc4bb4b07509758fb4d3bb', 'request_id': 'req-0d926c40-6711-4741-be41-92b323960af2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:43.952671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada222de75bc4bb4b07509758fb4d3bb', 'request_id': 'req-0d926c40-6711-4741-be41-92b323960af2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.036341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] The token cache is being invalidate because project ada222de75bc4bb4b07509758fb4d3bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:44.036866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.037667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.039171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 995/1977] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:43 2022] DELETE /identity/v3/projects/ada222de75bc4bb4b07509758fb4d3bb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:44.043166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:44.054757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70fe12-4860-4202-a8eb-52046bd5f1a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:44.116952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:44.117652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:44.117943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:44.118221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] PATH_INFO: `/v3/users/cbd61ac150374c8f9e1465d56734d4ed` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:44.119346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd61ac150374c8f9e1465d56734d4ed)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:44.121319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:44.340295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd61ac150374c8f9e1465d56734d4ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.340954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd61ac150374c8f9e1465d56734d4ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.410876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd61ac150374c8f9e1465d56734d4ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.411625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd61ac150374c8f9e1465d56734d4ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.426586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7feaba65-a965-4792-8293-c56c0b102ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd61ac150374c8f9e1465d56734d4ed'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:44.439839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 983/1978] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:44 2022] DELETE /identity/v3/users/cbd61ac150374c8f9e1465d56734d4ed => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:45.047092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:45.051120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:45.068945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d926c40-6711-4741-be41-92b323960af2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:45.142792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:45.143449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:45.143790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:45.144125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] PATH_INFO: `/v3/projects/3c2e368762764bbb8a0d62f7e34bc8e6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:45.145349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c2e368762764bbb8a0d62f7e34bc8e6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:45.147398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:45.245884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c2e368762764bbb8a0d62f7e34bc8e6', 'request_id': 'req-1d99cdcf-f59d-4ab8-858c-545808068056'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:45.252339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c2e368762764bbb8a0d62f7e34bc8e6', 'request_id': 'req-1d99cdcf-f59d-4ab8-858c-545808068056'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:45.258620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c2e368762764bbb8a0d62f7e34bc8e6', 'request_id': 'req-1d99cdcf-f59d-4ab8-858c-545808068056'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:45.341534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] The token cache is being invalidate because project 3c2e368762764bbb8a0d62f7e34bc8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:45.341874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:45.342504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d99cdcf-f59d-4ab8-858c-545808068056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:45.344407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 996/1979] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:45 2022] DELETE /identity/v3/projects/3c2e368762764bbb8a0d62f7e34bc8e6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:45.351268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec13935-99ad-4f8f-929a-31d947968ffd None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:45.351768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec13935-99ad-4f8f-929a-31d947968ffd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:45.351948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec13935-99ad-4f8f-929a-31d947968ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:45.372838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dec13935-99ad-4f8f-929a-31d947968ffd None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:45.374101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dec13935-99ad-4f8f-929a-31d947968ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:45.383665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:45.438146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 984/1980] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:45.447309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c218309-4c94-4aa7-aef3-7918ee31f9be None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:45.447674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c218309-4c94-4aa7-aef3-7918ee31f9be None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:45.448452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c218309-4c94-4aa7-aef3-7918ee31f9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:45.474117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6c218309-4c94-4aa7-aef3-7918ee31f9be None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:45.476551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6c218309-4c94-4aa7-aef3-7918ee31f9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:45.485193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 997/1981] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:45.552355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec13935-99ad-4f8f-929a-31d947968ffd None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:45.558786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec13935-99ad-4f8f-929a-31d947968ffd None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:45.564090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:45.564688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:45.564970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:45.565240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:45.565918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:45.566215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:45.567793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:45.579771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 985/1982] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:45.584225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c218309-4c94-4aa7-aef3-7918ee31f9be None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:45.593775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c218309-4c94-4aa7-aef3-7918ee31f9be None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:45.600816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:45.600816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:45.600816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:45.600816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:45.601464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:45.601987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:45.892478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 998/1983] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:45 2022] POST /identity/v3/projects => generated 367 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:45.893751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:45.903817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9885-ff87-4663-a523-12a9e7b0c3fb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:45.964315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:45.964959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:45.965293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:45.965591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:45.966236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:45.968205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:46.122354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 986/1984] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:45 2022] POST /identity/v3/users => generated 284 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:46.126637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:46.143847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613b9f6-f4d0-45c0-9acb-1be19dd767e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:46.150276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:46.152259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:46.152259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:46.152259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:46.152629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:46.154466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:46.167158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 999/1985] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:46.173308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:46.184959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07c54c-779c-40a8-a241-5ba7106a49bc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:46.191315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d70aff-75ab-4e40-bcaa-36ada364a57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:46.192284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d70aff-75ab-4e40-bcaa-36ada364a57d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:46.192284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d70aff-75ab-4e40-bcaa-36ada364a57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:46.192689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d70aff-75ab-4e40-bcaa-36ada364a57d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:46.193108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d70aff-75ab-4e40-bcaa-36ada364a57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:46.195023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d70aff-75ab-4e40-bcaa-36ada364a57d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:46.219799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 987/1986] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:46.223825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:46.234575 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454865b-3b65-4a48-8103-9098f0803f58 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:46.239292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:46.239959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:46.240242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:46.240520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] PATH_INFO: `/v3/projects/24e5ac932ca844faab63b97f9a17d195/users/73e259a38f1f449b978e234b7d190cdf/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:46.257659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e5ac932ca844faab63b97f9a17d195, user_id=73e259a38f1f449b978e234b7d190cdf, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:46.260140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:46.367827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1000/1987] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:46 2022] PUT /identity/v3/projects/24e5ac932ca844faab63b97f9a17d195/users/73e259a38f1f449b978e234b7d190cdf/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:46.370257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465602c-ba0b-48e5-9b84-d8e3945dd952 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:46.370775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465602c-ba0b-48e5-9b84-d8e3945dd952 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:46.371033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465602c-ba0b-48e5-9b84-d8e3945dd952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:46.425474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a465602c-ba0b-48e5-9b84-d8e3945dd952 None None] MFA Rules not processed for user `73e259a38f1f449b978e234b7d190cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:46.427713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a465602c-ba0b-48e5-9b84-d8e3945dd952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:46.555623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 988/1988] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:46.561235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:46.570711 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:46.652090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119bf37-3878-4713-ae89-fd55dd3e4dba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:46.659502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:46.661748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:46.661748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:46.661748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:46.661748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:46.663936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:46.810413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1001/1989] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:46 2022] POST /identity/v3/projects => generated 365 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:46.815357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465602c-ba0b-48e5-9b84-d8e3945dd952 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:46.841608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465602c-ba0b-48e5-9b84-d8e3945dd952 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:46.852579 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:46.929109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:46.929763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:46.930045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:46.930316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:46.931736 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:46.933854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:47.069375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 989/1990] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:46 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:47.070330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:47.085447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510b377-f13b-4f4e-a31e-e8698e52a30d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:47.107473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:47.108443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:47.108734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:47.109601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:47.110232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:47.112972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:47.136423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1002/1991] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:47.145546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:47.177369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c2bb7-5a5d-473d-bfcb-f4858bf616fc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:47.190228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:47.190228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:47.190228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:47.190228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] PATH_INFO: `/v3/projects/283d097b170f47e2bd6568dbaa4c1bce/users/79ae0a92dfac42c98c7b48eeef861bcc/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:47.207498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=283d097b170f47e2bd6568dbaa4c1bce, user_id=79ae0a92dfac42c98c7b48eeef861bcc, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:47.210520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:47.295293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 990/1992] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:47 2022] PUT /identity/v3/projects/283d097b170f47e2bd6568dbaa4c1bce/users/79ae0a92dfac42c98c7b48eeef861bcc/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:47.299414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:47.323444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582c263-ff28-40d6-8772-429d96c051db admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:47.449281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb49be-d1b9-45bc-9a5f-1a59d24a91c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:47.450011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb49be-d1b9-45bc-9a5f-1a59d24a91c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:47.450300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb49be-d1b9-45bc-9a5f-1a59d24a91c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:47.450633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb49be-d1b9-45bc-9a5f-1a59d24a91c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:47.451300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb49be-d1b9-45bc-9a5f-1a59d24a91c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:47.453472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb49be-d1b9-45bc-9a5f-1a59d24a91c6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:47.489835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1003/1993] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:47.493863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:47.509009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b04681-ecb7-4a06-8301-7ab63d1eac24 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:47.523312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:47.524338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:47.524549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:47.524838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] PATH_INFO: `/v3/projects/283d097b170f47e2bd6568dbaa4c1bce/users/79ae0a92dfac42c98c7b48eeef861bcc/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:47.543316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=283d097b170f47e2bd6568dbaa4c1bce, user_id=79ae0a92dfac42c98c7b48eeef861bcc, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:47.547516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:47.574168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 991/1994] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:47 2022] PUT /identity/v3/projects/283d097b170f47e2bd6568dbaa4c1bce/users/79ae0a92dfac42c98c7b48eeef861bcc/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:47.585204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596035f5-6cee-42b4-870b-43402ee56d88 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:47.585204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596035f5-6cee-42b4-870b-43402ee56d88 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:47.587740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596035f5-6cee-42b4-870b-43402ee56d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:47.638255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-596035f5-6cee-42b4-870b-43402ee56d88 None None] MFA Rules not processed for user `79ae0a92dfac42c98c7b48eeef861bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:47.646329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-596035f5-6cee-42b4-870b-43402ee56d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:47.789746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:47.798058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1004/1995] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:47.879206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:47.883337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:47.980444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707c98c-b7a8-45dd-b7e7-641851edb1c5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:47.986337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:47.986967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:47.987265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:47.987553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:47.997239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:48.003894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:48.011648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 992/1996] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:48.331677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596035f5-6cee-42b4-870b-43402ee56d88 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:48.340033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596035f5-6cee-42b4-870b-43402ee56d88 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:48.344703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b07276-0948-42d7-b4c3-575331507f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:48.345462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b07276-0948-42d7-b4c3-575331507f79 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:48.345848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b07276-0948-42d7-b4c3-575331507f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:48.346191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b07276-0948-42d7-b4c3-575331507f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:48.406830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b07276-0948-42d7-b4c3-575331507f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:48.408786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b07276-0948-42d7-b4c3-575331507f79 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:48.420714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1005/1997] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:50.814342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:50.816201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:50.895071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f950-5fc1-472d-aa76-644b1785b912 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:50.901279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:50.902048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:50.902402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:50.902674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] PATH_INFO: `/v3/users/82d101b28c5448248a03ab4d370a6602` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:50.903814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82d101b28c5448248a03ab4d370a6602)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:50.907329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:50.914031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:51.129161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82d101b28c5448248a03ab4d370a6602'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.129873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82d101b28c5448248a03ab4d370a6602'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.189490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82d101b28c5448248a03ab4d370a6602'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.190166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82d101b28c5448248a03ab4d370a6602'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.204033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82d101b28c5448248a03ab4d370a6602'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.216873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 993/1998] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:50 2022] DELETE /identity/v3/users/82d101b28c5448248a03ab4d370a6602 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:51.224209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:51.224209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:51.224209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:51.246393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:51.248983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:51.331789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1006/1999] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:51.341809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:51.343332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:51.402900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c7d58-a766-4868-a3e6-67f953b424e9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:51.407223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:51.407846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:51.408161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:51.408437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:51.419462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:51.420139 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:51.429950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 994/2000] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:51.552675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:51.554126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:51.566748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882dc07-6f52-44d9-b541-57a22cb81dc8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:51.571282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:51.571909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:51.573936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:51.573936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] PATH_INFO: `/v3/users/8d2934973ef44a3492db375a8a5c4fa9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:51.573936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2934973ef44a3492db375a8a5c4fa9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:51.576015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:51.827260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2934973ef44a3492db375a8a5c4fa9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.828875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2934973ef44a3492db375a8a5c4fa9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.845645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2934973ef44a3492db375a8a5c4fa9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.899903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:51.901414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:51.924626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45d4bc-392a-4aa1-a66d-e02647bafffe service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:51.933249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2934973ef44a3492db375a8a5c4fa9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.934187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7d39f4c-d972-4c97-846e-ee973f2a6898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2934973ef44a3492db375a8a5c4fa9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:51.934893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:51.935761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:51.936135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:51.936422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] PATH_INFO: `/v3/projects/84f875c138cc4d3c915351cbbf58f559` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:51.938400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f875c138cc4d3c915351cbbf58f559)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:51.940956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:51.957725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1007/2001] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:51 2022] DELETE /identity/v3/users/8d2934973ef44a3492db375a8a5c4fa9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:51.962112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac7559-c336-473b-981f-253757be73ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:51.962484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac7559-c336-473b-981f-253757be73ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:51.962835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac7559-c336-473b-981f-253757be73ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:51.988091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9cac7559-c336-473b-981f-253757be73ad None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:51.989644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9cac7559-c336-473b-981f-253757be73ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:51.996383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:52.069645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f875c138cc4d3c915351cbbf58f559', 'request_id': 'req-0aba6239-7d6c-4e8b-a913-17cc65d034e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.121099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1008/2002] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:52.128823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac7559-c336-473b-981f-253757be73ad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:52.130426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9cac7559-c336-473b-981f-253757be73ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:52.208145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f875c138cc4d3c915351cbbf58f559', 'request_id': 'req-0aba6239-7d6c-4e8b-a913-17cc65d034e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.223522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f875c138cc4d3c915351cbbf58f559', 'request_id': 'req-0aba6239-7d6c-4e8b-a913-17cc65d034e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.237162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac7559-c336-473b-981f-253757be73ad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:52.237880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] The token cache is being invalidate because project 84f875c138cc4d3c915351cbbf58f559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:52.238175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.239238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.244327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 995/2003] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:51 2022] DELETE /identity/v3/projects/84f875c138cc4d3c915351cbbf58f559 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:52.245291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:52.246749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:52.268214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:52.307128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:52.307872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:52.309706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:52.310185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:52.317762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:52.369476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba6239-7d6c-4e8b-a913-17cc65d034e8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:52.376099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:52.382414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:52.382414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:52.382414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] PATH_INFO: `/v3/users/e7fbb148c2344dcabd2063b60c841e12` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:52.382414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7fbb148c2344dcabd2063b60c841e12)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:52.382414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:52.479162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:52.480758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:52.492451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1009/2004] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:52.588822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7fbb148c2344dcabd2063b60c841e12'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.589975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7fbb148c2344dcabd2063b60c841e12'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.714127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7fbb148c2344dcabd2063b60c841e12'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.717710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7fbb148c2344dcabd2063b60c841e12'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.732412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7fbb148c2344dcabd2063b60c841e12'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:52.753277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 996/2005] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:52 2022] DELETE /identity/v3/users/e7fbb148c2344dcabd2063b60c841e12 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:53.085695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:53.087686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:53.106358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00042e1-73ef-4c61-af26-428b0721727d service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:53.137784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:53.214510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:53.215167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:53.215475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:53.215760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] PATH_INFO: `/v3/projects/30dfdaf04fae4c4dbd97043024d6d4be` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:53.217063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30dfdaf04fae4c4dbd97043024d6d4be)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:53.219872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:53.372740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30dfdaf04fae4c4dbd97043024d6d4be', 'request_id': 'req-0f28f137-2ad9-4d8f-8422-2a519de989a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.378291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30dfdaf04fae4c4dbd97043024d6d4be', 'request_id': 'req-0f28f137-2ad9-4d8f-8422-2a519de989a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.384428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30dfdaf04fae4c4dbd97043024d6d4be', 'request_id': 'req-0f28f137-2ad9-4d8f-8422-2a519de989a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.393310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:53.396902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:53.433233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742f3c4-d3d2-43a1-b191-4c9327b08b52 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:53.518125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:53.518856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:53.519797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:53.520172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] PATH_INFO: `/v3/projects/c1136f95971e4e1983300e81b5d469c4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:53.521580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1136f95971e4e1983300e81b5d469c4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:53.524185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:53.527260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] The token cache is being invalidate because project 30dfdaf04fae4c4dbd97043024d6d4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:53.532884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.532884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.532884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1010/2006] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:53 2022] DELETE /identity/v3/projects/30dfdaf04fae4c4dbd97043024d6d4be => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:53.534849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:53.536324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:53.562400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28f137-2ad9-4d8f-8422-2a519de989a6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:53.568040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:53.569117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:53.569410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:53.569726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] PATH_INFO: `/v3/users/a76db9c53ed545869cb3532868e165ee` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:53.571393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76db9c53ed545869cb3532868e165ee)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:53.573364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:53.646460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:53.660186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1136f95971e4e1983300e81b5d469c4', 'request_id': 'req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.750249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1136f95971e4e1983300e81b5d469c4', 'request_id': 'req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.756663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1136f95971e4e1983300e81b5d469c4', 'request_id': 'req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.769151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] The token cache is being invalidate because project c1136f95971e4e1983300e81b5d469c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:53.769445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.770132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c2eb06ae-1409-4015-a8d4-aa6daffebb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.771623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 997/2007] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:53 2022] DELETE /identity/v3/projects/c1136f95971e4e1983300e81b5d469c4 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:53.776473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee09aa-86fe-42b6-9a5a-6fcb43db6ee3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:53.777168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee09aa-86fe-42b6-9a5a-6fcb43db6ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:53.777168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee09aa-86fe-42b6-9a5a-6fcb43db6ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:53.792313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76db9c53ed545869cb3532868e165ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.793048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76db9c53ed545869cb3532868e165ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.800774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7eee09aa-86fe-42b6-9a5a-6fcb43db6ee3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:53.802036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7eee09aa-86fe-42b6-9a5a-6fcb43db6ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:53.809886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76db9c53ed545869cb3532868e165ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.871375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 998/2008] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:53.879422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb7542-d865-484b-b8f2-02bc9782eb48 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:53.879716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb7542-d865-484b-b8f2-02bc9782eb48 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:53.879989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb7542-d865-484b-b8f2-02bc9782eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:53.886596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76db9c53ed545869cb3532868e165ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.887795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76db9c53ed545869cb3532868e165ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:53.901269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-73bb7542-d865-484b-b8f2-02bc9782eb48 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:53.902159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1011/2009] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:53 2022] DELETE /identity/v3/users/a76db9c53ed545869cb3532868e165ee => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:53.902656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-73bb7542-d865-484b-b8f2-02bc9782eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:53.992513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 999/2010] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:53.997227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.010468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66062ab0-900d-458c-9f75-b4675e6f5256 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.016035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.016353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.016680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.016950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.017822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:54.018121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.019740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.033104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1012/2011] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:14:54.037442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb7542-d865-484b-b8f2-02bc9782eb48 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.047236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb7542-d865-484b-b8f2-02bc9782eb48 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.051921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.052517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.052796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.053067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.053758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.055768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.190310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1000/2012] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:54.192312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.204667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63574ec-9b73-4d54-9d31-e32553ce0138 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.209194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:54.265649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.266371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.266659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.266954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.267584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.269614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.299821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.301788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:54.317517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec63e6-9121-4e62-93d6-6be940644dc0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.327154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.328464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.328747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.329020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] PATH_INFO: `/v3/projects/f08f40f41d4c4612806feb672a9c4b55` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.330244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f08f40f41d4c4612806feb672a9c4b55)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.332211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.345255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1013/2013] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:54.349283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.360062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37778fda-3402-4dc8-9ef5-2daa50a90084 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.364820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.365634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.365926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.366201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.366871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.368478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.379037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1014/2014] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:54.383055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.393352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc7fb7-a927-4bf2-85b1-5a093117d107 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.398027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.398602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.398885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.399163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.399822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.401388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.412048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1015/2015] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:54.416109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.426916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ce2b4-d854-40bb-9371-0ca7189b1261 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.431786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d6275-19bf-4c9f-9d91-f2c81087cf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.432433 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6275-19bf-4c9f-9d91-f2c81087cf1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.432721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6275-19bf-4c9f-9d91-f2c81087cf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.433147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6275-19bf-4c9f-9d91-f2c81087cf1c admin admin] PATH_INFO: `/v3/projects/2e0afb6c943047b3a459c7cabfee9c48/users/5fd0f3a5ff4c4dcf81978d55795fcd50/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.435500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:54.448696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d6275-19bf-4c9f-9d91-f2c81087cf1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0afb6c943047b3a459c7cabfee9c48, user_id=5fd0f3a5ff4c4dcf81978d55795fcd50, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.450812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f08f40f41d4c4612806feb672a9c4b55', 'request_id': 'req-71dce125-c2c3-4b88-b0b3-06d7b8b50042'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:54.451398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d6275-19bf-4c9f-9d91-f2c81087cf1c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.532765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f08f40f41d4c4612806feb672a9c4b55', 'request_id': 'req-71dce125-c2c3-4b88-b0b3-06d7b8b50042'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:54.540200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f08f40f41d4c4612806feb672a9c4b55', 'request_id': 'req-71dce125-c2c3-4b88-b0b3-06d7b8b50042'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:54.540822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1016/2016] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:54 2022] PUT /identity/v3/projects/2e0afb6c943047b3a459c7cabfee9c48/users/5fd0f3a5ff4c4dcf81978d55795fcd50/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:54.552825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.553115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.553639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.553751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] The token cache is being invalidate because project f08f40f41d4c4612806feb672a9c4b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:54.553853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:54.554503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-71dce125-c2c3-4b88-b0b3-06d7b8b50042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:54.555997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1001/2017] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:54 2022] DELETE /identity/v3/projects/f08f40f41d4c4612806feb672a9c4b55 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:54.560845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536a5b-8deb-46eb-aced-a1d7afc6886f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.561274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536a5b-8deb-46eb-aced-a1d7afc6886f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.561585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536a5b-8deb-46eb-aced-a1d7afc6886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.582006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2a536a5b-8deb-46eb-aced-a1d7afc6886f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:54.583233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2a536a5b-8deb-46eb-aced-a1d7afc6886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:54.583853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] MFA Rules not processed for user `5fd0f3a5ff4c4dcf81978d55795fcd50`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:54.585050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:54.644990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1002/2018] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:54.655741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1017/2019] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:54.655938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d9d76-acbc-4c80-85b8-0cf1903d357c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.655938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d9d76-acbc-4c80-85b8-0cf1903d357c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.655938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d9d76-acbc-4c80-85b8-0cf1903d357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.656703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.658068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:54.671402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86417b5-20b9-4086-bd7a-a208c28b9424 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.673328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3f3d9d76-acbc-4c80-85b8-0cf1903d357c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:54.674623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3f3d9d76-acbc-4c80-85b8-0cf1903d357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:54.676062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.678504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.678504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.678504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.678504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.683798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.684945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1003/2020] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:54.689443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d9d76-acbc-4c80-85b8-0cf1903d357c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.698730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d9d76-acbc-4c80-85b8-0cf1903d357c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.704222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.704830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.705165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.705437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.706125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:54.706427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.708014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.721048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1004/2021] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:14:54.725314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.735492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239f9a5-9bf2-49a4-b458-bc3c921def8d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.740417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.740942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.741222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.741526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.742189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.744084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.784581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1018/2022] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/projects => generated 393 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:54.788428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.812590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:54.836024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1005/2023] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/projects => generated 361 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:54.840226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.874141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f5e93-91a0-4894-95d6-00b1021e4211 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.904424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafe552-0840-44a4-a7b1-d97ae0c03d20 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.910282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.910896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.911219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.911495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.912126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.914071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.935441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.936069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.936411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.936725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.937442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:54.939569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:54.978293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1006/2024] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/users => generated 281 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:54.982257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:54.992589 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929683b8-041c-4fde-9c87-7fa01ae0c439 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:54.997405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:54.998050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:54.998346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:54.998634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:54.999325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.001003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.003655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:55.014024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1007/2025] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:55.018734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.028894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec824d-c43f-4fca-862b-94af2b598fc7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.040768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.041396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.041704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.041981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.042633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.044284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.049796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1019/2026] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:54 2022] POST /identity/v3/users => generated 303 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:55.053196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.059035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1008/2027] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:55.063486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.065631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f308567-f6b5-480b-affd-8ee423a2297b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.071085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.071769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.072049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.072331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.072957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.073428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef5f31-e875-4c27-9507-3578cc9a8f76 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.074916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.079470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec6f5e-b8c5-4173-9155-bacb9a05d466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.080174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec6f5e-b8c5-4173-9155-bacb9a05d466 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.080458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec6f5e-b8c5-4173-9155-bacb9a05d466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.080738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec6f5e-b8c5-4173-9155-bacb9a05d466 admin admin] PATH_INFO: `/v3/projects/17c95faefc354571b76b0ceec69d99f8/users/2c99ea8cd59f411fbc208dd1144f18c1/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.086942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1020/2028] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:55.098293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec6f5e-b8c5-4173-9155-bacb9a05d466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c95faefc354571b76b0ceec69d99f8, user_id=2c99ea8cd59f411fbc208dd1144f18c1, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.100976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec6f5e-b8c5-4173-9155-bacb9a05d466 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.103447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.113530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d316da-144c-4e9b-b229-cfc9539566b1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.118330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.118988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.119267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.119537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] PATH_INFO: `/v3/projects/c9c25dee560e4ee39e8b09f4dd451f92/users/f4095573a757470cacb496aa64f0f8a9/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.134064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c25dee560e4ee39e8b09f4dd451f92, user_id=f4095573a757470cacb496aa64f0f8a9, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.136806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.195513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1009/2029] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:55 2022] PUT /identity/v3/projects/17c95faefc354571b76b0ceec69d99f8/users/2c99ea8cd59f411fbc208dd1144f18c1/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:55.208041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715e125-ecaf-4ab7-8486-f99d070e6b49 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.208332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715e125-ecaf-4ab7-8486-f99d070e6b49 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.208640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715e125-ecaf-4ab7-8486-f99d070e6b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.226263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1021/2030] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:55 2022] PUT /identity/v3/projects/c9c25dee560e4ee39e8b09f4dd451f92/users/f4095573a757470cacb496aa64f0f8a9/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:55.230747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.242581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3715e125-ecaf-4ab7-8486-f99d070e6b49 None None] MFA Rules not processed for user `2c99ea8cd59f411fbc208dd1144f18c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:55.243859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3715e125-ecaf-4ab7-8486-f99d070e6b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:55.313341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3b2be-dcc7-45d6-947c-9a23d803985b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.318296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.318923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.319203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.319480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.320141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.321823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.326947 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1010/2031] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:55.334693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1022/2032] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:55.339652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715e125-ecaf-4ab7-8486-f99d070e6b49 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.352607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715e125-ecaf-4ab7-8486-f99d070e6b49 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.363795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fbb0-7426-4408-9bbf-2c9fcbf1d9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.364495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fbb0-7426-4408-9bbf-2c9fcbf1d9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.364781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fbb0-7426-4408-9bbf-2c9fcbf1d9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.365093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fbb0-7426-4408-9bbf-2c9fcbf1d9f6 admin admin] PATH_INFO: `/v3/projects/c9c25dee560e4ee39e8b09f4dd451f92/users/f4095573a757470cacb496aa64f0f8a9/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.367316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fbb0-7426-4408-9bbf-2c9fcbf1d9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c25dee560e4ee39e8b09f4dd451f92, user_id=f4095573a757470cacb496aa64f0f8a9, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.370155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fbb0-7426-4408-9bbf-2c9fcbf1d9f6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.370485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.372412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:55.383506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1011/2033] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:55 2022] PUT /identity/v3/projects/c9c25dee560e4ee39e8b09f4dd451f92/users/f4095573a757470cacb496aa64f0f8a9/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:55.395610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80f02-7752-4502-be03-0d44e830755c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.395938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80f02-7752-4502-be03-0d44e830755c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.396596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80f02-7752-4502-be03-0d44e830755c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.401490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:55.429704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-52d80f02-7752-4502-be03-0d44e830755c None None] MFA Rules not processed for user `f4095573a757470cacb496aa64f0f8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:55.431281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-52d80f02-7752-4502-be03-0d44e830755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:55.453306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d38cf5-dcee-4907-9a81-855f037a3640 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.518240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.518866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.519151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.519429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.520940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1012/2034] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:55.540940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80f02-7752-4502-be03-0d44e830755c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.542335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-52d80f02-7752-4502-be03-0d44e830755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:55.554310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80f02-7752-4502-be03-0d44e830755c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.557952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.558559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.558984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.559577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.569987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.571449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.581950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1013/2035] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:55.583097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.584503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.593666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1023/2036] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:55.610057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.624504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:55.658576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.667162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775795e-b6b7-4b1f-9998-0014ec1df53c service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.671015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.672223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.672498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.672770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.695476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e9b7d-c604-499f-9a88-7a4d42fe5265 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.700607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.700933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.701218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.701500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] PATH_INFO: `/v3/users/2c99ea8cd59f411fbc208dd1144f18c1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.703102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c99ea8cd59f411fbc208dd1144f18c1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.705098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.736512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.738003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.747119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1024/2037] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:55.764740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:55.767109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:55.781871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bb8f3-0169-4741-96fb-36ffb08da6bf service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:55.787078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:55.787634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:55.787938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:55.788197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] PATH_INFO: `/v3/users/73e259a38f1f449b978e234b7d190cdf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:55.789348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e259a38f1f449b978e234b7d190cdf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:55.791315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:55.815602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:55.932825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c99ea8cd59f411fbc208dd1144f18c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:55.933488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c99ea8cd59f411fbc208dd1144f18c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.038432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c99ea8cd59f411fbc208dd1144f18c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.039636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c99ea8cd59f411fbc208dd1144f18c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.050061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e259a38f1f449b978e234b7d190cdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.050910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e259a38f1f449b978e234b7d190cdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.054202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-91a3f8d4-f108-46d6-ba97-d98c8917882d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c99ea8cd59f411fbc208dd1144f18c1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.064324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e259a38f1f449b978e234b7d190cdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.070715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1014/2038] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:55 2022] DELETE /identity/v3/users/2c99ea8cd59f411fbc208dd1144f18c1 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:56.072216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:56.072506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:56.072864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:56.092369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:56.093902 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:56.156451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1015/2039] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:56.158818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e259a38f1f449b978e234b7d190cdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.159421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-18a42845-1f06-4a32-a080-980ecf96c5ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e259a38f1f449b978e234b7d190cdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:56.167377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:56.168907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:56.179938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1025/2040] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:55 2022] DELETE /identity/v3/users/73e259a38f1f449b978e234b7d190cdf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:56.182090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:56.182647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:56.195141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:56.207123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:56.208059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:56.219274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:56.241603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc40a12-ebc8-46b9-8715-ce5621679a30 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:56.270246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1026/2041] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:56.280004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:56.281381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:56.292564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:56.293483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:56.295155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:56.295155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:56.303954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:56.305453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:56.315835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1016/2042] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:56.340973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d603-9d87-4856-9b44-2e3b12890501 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:56.345317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:56.345461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:56.345713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:56.345994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:56.361586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:56.361586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:56.370057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1027/2043] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:14:56.883007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:56.887088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:56.889074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:56.890537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:56.908523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80360c7-ea53-4e12-ade9-aa7a98a7eb2a service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:56.908886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd090ec9-5f36-4ae2-b06c-99277fe03315 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:56.911300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:56.912010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:56.912255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:56.912537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] PATH_INFO: `/v3/projects/24e5ac932ca844faab63b97f9a17d195` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:56.913695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:56.913833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e5ac932ca844faab63b97f9a17d195)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:56.914341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:56.914624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:56.914900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] PATH_INFO: `/v3/projects/17c95faefc354571b76b0ceec69d99f8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:56.915906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:56.916062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c95faefc354571b76b0ceec69d99f8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:56.918291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.048114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:57.059002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c95faefc354571b76b0ceec69d99f8', 'request_id': 'req-e485d71c-2ccc-410f-8db9-7d750d165de2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.061706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e5ac932ca844faab63b97f9a17d195', 'request_id': 'req-a7ee030e-3742-49a6-b359-170b999b09e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.068772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e5ac932ca844faab63b97f9a17d195', 'request_id': 'req-a7ee030e-3742-49a6-b359-170b999b09e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.076276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e5ac932ca844faab63b97f9a17d195', 'request_id': 'req-a7ee030e-3742-49a6-b359-170b999b09e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.132749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c95faefc354571b76b0ceec69d99f8', 'request_id': 'req-e485d71c-2ccc-410f-8db9-7d750d165de2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.144674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c95faefc354571b76b0ceec69d99f8', 'request_id': 'req-e485d71c-2ccc-410f-8db9-7d750d165de2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.152679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] The token cache is being invalidate because project 17c95faefc354571b76b0ceec69d99f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:57.152972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.153782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e485d71c-2ccc-410f-8db9-7d750d165de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.155285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1017/2044] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:56 2022] DELETE /identity/v3/projects/17c95faefc354571b76b0ceec69d99f8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:57.161788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b1eee-8469-483c-b4aa-9888d3362495 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.162086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b1eee-8469-483c-b4aa-9888d3362495 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.162346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b1eee-8469-483c-b4aa-9888d3362495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.190225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cf9b1eee-8469-483c-b4aa-9888d3362495 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:57.191468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cf9b1eee-8469-483c-b4aa-9888d3362495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:57.246085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] The token cache is being invalidate because project 24e5ac932ca844faab63b97f9a17d195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:57.246361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.246982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.248475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1028/2045] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:56 2022] DELETE /identity/v3/projects/24e5ac932ca844faab63b97f9a17d195 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:57.252494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:57.253961 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:57.258298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1018/2046] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:57.268131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee030e-3742-49a6-b359-170b999b09e5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:57.270117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565239-d2c3-4c7d-8051-216662f15a61 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.270416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565239-d2c3-4c7d-8051-216662f15a61 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.270692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565239-d2c3-4c7d-8051-216662f15a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.273182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:57.273936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.274253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.274532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] PATH_INFO: `/v3/users/79ae0a92dfac42c98c7b48eeef861bcc` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.275671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ae0a92dfac42c98c7b48eeef861bcc)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:57.277698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.290388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-42565239-d2c3-4c7d-8051-216662f15a61 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:57.291640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-42565239-d2c3-4c7d-8051-216662f15a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:57.300285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1019/2047] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:14:57.304395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565239-d2c3-4c7d-8051-216662f15a61 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:57.314423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565239-d2c3-4c7d-8051-216662f15a61 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:57.319132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:57.319733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.320014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.320290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.320936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:57.321243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:57.322876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.324510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:57.338357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1020/2048] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:14:57.339381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:57.349325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8124a25-4b60-46bc-b56f-ee79b24d28e2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:57.354419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:57.355042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.355325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.355593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.356237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:57.358226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.475125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1021/2049] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:57 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:57.479310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:57.483315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ae0a92dfac42c98c7b48eeef861bcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.484690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ae0a92dfac42c98c7b48eeef861bcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.517063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ae0a92dfac42c98c7b48eeef861bcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.580538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0f754-42f4-432b-9c93-3fbd74f42a89 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:57.583408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ae0a92dfac42c98c7b48eeef861bcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.584224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ae0a92dfac42c98c7b48eeef861bcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:57.586970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:57.592077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.592077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.592077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.592077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:57.592077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.597975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1029/2050] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:14:57 2022] DELETE /identity/v3/users/79ae0a92dfac42c98c7b48eeef861bcc => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:57.676851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1022/2051] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:57 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:14:57.680882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:57.682321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:57.747488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce9322-1835-498a-b7c9-b71d59b5a46e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:57.752591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:57.753209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.753499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.753801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.754453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:57.756068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.766931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1030/2052] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:57.771106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:57.781683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228348d-6d88-450a-9627-b31643ae55d4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:57.786789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f006-2f77-4f85-8944-96488a504c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:57.787470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f006-2f77-4f85-8944-96488a504c9c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.787760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f006-2f77-4f85-8944-96488a504c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.788033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f006-2f77-4f85-8944-96488a504c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.788669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f006-2f77-4f85-8944-96488a504c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:57.790296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f006-2f77-4f85-8944-96488a504c9c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.802267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1023/2053] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:57.806435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:57.819687 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a9399-cd82-43f0-bec2-4d1b21ff51ce admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:57.837756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:57.838871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.838871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.839081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] PATH_INFO: `/v3/projects/9f1ae25e634d4380a6933101fb94ad2e/users/a8925da59209424fbd6375f056aee752/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.855400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1ae25e634d4380a6933101fb94ad2e, user_id=a8925da59209424fbd6375f056aee752, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:57.858193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:57.934656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1031/2054] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:57 2022] PUT /identity/v3/projects/9f1ae25e634d4380a6933101fb94ad2e/users/a8925da59209424fbd6375f056aee752/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:14:57.940562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:57.941476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:57.941476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:57.966309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:57.968339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:58.042710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1024/2055] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:58.053551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:58.055583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:58.092717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:58.094776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:58.107340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3630365-d10e-49df-8c8e-9f2abfbee972 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:58.115771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:58.115771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:58.115771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:58.116103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] PATH_INFO: `/v3/projects/283d097b170f47e2bd6568dbaa4c1bce` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:58.117401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=283d097b170f47e2bd6568dbaa4c1bce)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:58.119407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:58.122949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda3950-3493-48a1-afe1-e5945d75c7c7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:58.126250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b7043-a1d5-444a-9615-8b39729a71ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:58.126927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b7043-a1d5-444a-9615-8b39729a71ba service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:58.127280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b7043-a1d5-444a-9615-8b39729a71ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:58.128051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b7043-a1d5-444a-9615-8b39729a71ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:58.137676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b7043-a1d5-444a-9615-8b39729a71ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:58.139125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b7043-a1d5-444a-9615-8b39729a71ba service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:58.155282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1032/2056] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:14:58.239062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '283d097b170f47e2bd6568dbaa4c1bce', 'request_id': 'req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:58.267927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:58.334661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '283d097b170f47e2bd6568dbaa4c1bce', 'request_id': 'req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:58.342023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '283d097b170f47e2bd6568dbaa4c1bce', 'request_id': 'req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:58.357721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] The token cache is being invalidate because project 283d097b170f47e2bd6568dbaa4c1bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:14:58.358012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:58.360331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1262e4a8-e397-49b8-8b7f-7e34eb92d4e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:14:58.365510 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1025/2057] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:14:58 2022] DELETE /identity/v3/projects/283d097b170f47e2bd6568dbaa4c1bce => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:58.382945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590010-803e-4957-bb1c-0fa6ae51b6ac None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:58.383461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590010-803e-4957-bb1c-0fa6ae51b6ac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:58.383826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590010-803e-4957-bb1c-0fa6ae51b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:58.410149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5f590010-803e-4957-bb1c-0fa6ae51b6ac None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:58.411625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5f590010-803e-4957-bb1c-0fa6ae51b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:58.423369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:58.503299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1033/2058] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:58.513388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0e1cc-906c-455a-9b7a-a15565ff6c5e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:58.513388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0e1cc-906c-455a-9b7a-a15565ff6c5e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:58.513687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0e1cc-906c-455a-9b7a-a15565ff6c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:58.534304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d8e0e1cc-906c-455a-9b7a-a15565ff6c5e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:58.535898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d8e0e1cc-906c-455a-9b7a-a15565ff6c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:58.543834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1026/2059] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:58.549325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590010-803e-4957-bb1c-0fa6ae51b6ac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:58.559538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590010-803e-4957-bb1c-0fa6ae51b6ac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:58.565758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:58.566766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:58.568778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:58.568778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:58.568778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:14:58.568778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:58.569851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:58.601420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1034/2060] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:14:58.603661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0e1cc-906c-455a-9b7a-a15565ff6c5e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:58.615079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0e1cc-906c-455a-9b7a-a15565ff6c5e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:58.620222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:58.620936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:58.630621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:58.630621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:58.630621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:58.630621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:58.787767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1027/2061] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:58 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:58.792300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:58.810413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41c147-d03f-4660-b3b9-bb4cc05b5693 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:58.917041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:58.917554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:58.917866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:58.918207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:58.918914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:58.920926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.053639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1035/2062] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:58 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:59.057798 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.070025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860fc2-1d01-473b-8709-cd532bac880b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.074916 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.075597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.078458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.078458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.078458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.079243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.090572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1028/2063] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:59.094478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.105455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3558691-9a2d-4a15-92bf-edce93574c1e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.111615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583d335-cf37-4648-b2a8-4f12067e3df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.112228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d335-cf37-4648-b2a8-4f12067e3df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.112510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d335-cf37-4648-b2a8-4f12067e3df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.112780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d335-cf37-4648-b2a8-4f12067e3df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.113480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583d335-cf37-4648-b2a8-4f12067e3df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.115522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583d335-cf37-4648-b2a8-4f12067e3df0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.131385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1036/2064] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:59.136399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.148799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2de915-a3ad-4b2a-96ae-4d64dddae4a2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.154345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.155018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.155304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.156254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] PATH_INFO: `/v3/projects/26c1939be17c457a853903373f769562/users/351ef66ee191480b8d36535bfc55e52f/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.178335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c1939be17c457a853903373f769562, user_id=351ef66ee191480b8d36535bfc55e52f, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.181036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.275224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1029/2065] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:59 2022] PUT /identity/v3/projects/26c1939be17c457a853903373f769562/users/351ef66ee191480b8d36535bfc55e52f/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:14:59.297754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82db9-a5b4-4bbd-811a-2a6f17ff633e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.297984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82db9-a5b4-4bbd-811a-2a6f17ff633e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.298267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82db9-a5b4-4bbd-811a-2a6f17ff633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.326398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aab82db9-a5b4-4bbd-811a-2a6f17ff633e None None] MFA Rules not processed for user `351ef66ee191480b8d36535bfc55e52f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:59.328208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aab82db9-a5b4-4bbd-811a-2a6f17ff633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:59.403510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1037/2066] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:59.410909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.480130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe794232-d126-4613-a652-7cd3ed988758 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.485465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4242b-096d-4144-ae65-db4ff871a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.486107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4242b-096d-4144-ae65-db4ff871a2af admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.486388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4242b-096d-4144-ae65-db4ff871a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.486694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4242b-096d-4144-ae65-db4ff871a2af admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.487329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4242b-096d-4144-ae65-db4ff871a2af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.489272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4242b-096d-4144-ae65-db4ff871a2af admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.573638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1030/2067] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:14:59 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:59.577966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82db9-a5b4-4bbd-811a-2a6f17ff633e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.589844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82db9-a5b4-4bbd-811a-2a6f17ff633e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.652740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:59.659489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.660157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.660448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.661810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.661810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.663763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.759232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369484b-da35-4b6d-a0d2-e1fa6bc7edf4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.759696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369484b-da35-4b6d-a0d2-e1fa6bc7edf4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.760063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369484b-da35-4b6d-a0d2-e1fa6bc7edf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.791079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9369484b-da35-4b6d-a0d2-e1fa6bc7edf4 None None] MFA Rules not processed for user `a8925da59209424fbd6375f056aee752`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:14:59.792677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9369484b-da35-4b6d-a0d2-e1fa6bc7edf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:59.843939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1038/2068] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:14:59 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:14:59.845193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.849158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:14:59.855562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd0736-b9cc-4018-8ba3-f86a2a798980 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.860530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.861140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.861421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.862735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.862735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.864418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.872618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1031/2069] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:14:59.876126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1039/2070] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:14:59.880285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369484b-da35-4b6d-a0d2-e1fa6bc7edf4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.884647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.886679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:14:59.891796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369484b-da35-4b6d-a0d2-e1fa6bc7edf4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.897329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.898038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.898393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.898763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.899514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.900998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.912899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1032/2071] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:14:59.916984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:14:59.926885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8034cc0-d14d-471b-b56f-c16e46d20962 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.931517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.932170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.932449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.932721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] PATH_INFO: `/v3/projects/2cb8628b6e79489eae93c648ccc7c43c/users/33b6f54f6d10462d9e6323166aafce0b/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.946364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceac9cb-22dd-411e-8add-24f3ede514a4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:14:59.948076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb8628b6e79489eae93c648ccc7c43c, user_id=33b6f54f6d10462d9e6323166aafce0b, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.950495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f081d7-2bd6-4717-b56c-809509a76e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:14:59.951247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f081d7-2bd6-4717-b56c-809509a76e18 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:14:59.951360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.951563 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f081d7-2bd6-4717-b56c-809509a76e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:14:59.951794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f081d7-2bd6-4717-b56c-809509a76e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:14:59.961717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f081d7-2bd6-4717-b56c-809509a76e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:14:59.963193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f081d7-2bd6-4717-b56c-809509a76e18 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:14:59.972470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1040/2072] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:14:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:00.013867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1033/2073] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:14:59 2022] PUT /identity/v3/projects/2cb8628b6e79489eae93c648ccc7c43c/users/33b6f54f6d10462d9e6323166aafce0b/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:00.025883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13b837-327e-4453-8172-54b5c9c5074c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.026233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13b837-327e-4453-8172-54b5c9c5074c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.026507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13b837-327e-4453-8172-54b5c9c5074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.058077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6d13b837-327e-4453-8172-54b5c9c5074c None None] MFA Rules not processed for user `33b6f54f6d10462d9e6323166aafce0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:00.059328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6d13b837-327e-4453-8172-54b5c9c5074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:00.143093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1041/2074] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:00.147492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.215892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3e39-c774-42ba-8793-580db39052b9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:00.222744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:00.223353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.223631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.223904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.224545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:00.226491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:00.315351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1034/2075] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:00 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:00.325539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13b837-327e-4453-8172-54b5c9c5074c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.338421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13b837-327e-4453-8172-54b5c9c5074c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:00.378386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:00.401434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:00.402108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.402396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.402673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.403316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:00.405336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:00.484601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1042/2076] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:00 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:00.485643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.504529 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f6b7-021b-4c87-ad3b-5cf43fcf098b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:00.509461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:00.510106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.510385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.510655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.511362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:00.513312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:00.524539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1035/2077] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:00.528841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.553802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03395cfe-d297-4e16-836c-4f3429cc19f5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:00.553802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:00.553802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.553802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.554150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] PATH_INFO: `/v3/projects/dae0c94e56334b4caf3d42f573c94ba8/users/d1c1d1e4a1e64944a219c244e6e36f3b/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.563296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae0c94e56334b4caf3d42f573c94ba8, user_id=d1c1d1e4a1e64944a219c244e6e36f3b, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:00.566689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:00.639814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1043/2078] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:00 2022] PUT /identity/v3/projects/dae0c94e56334b4caf3d42f573c94ba8/users/d1c1d1e4a1e64944a219c244e6e36f3b/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:00.644407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.657774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2210c9-92b1-4b28-8499-83755a0d2c50 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:00.722136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd6c4f-9636-453e-8a33-7840f100d543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:00.722762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6c4f-9636-453e-8a33-7840f100d543 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.723045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6c4f-9636-453e-8a33-7840f100d543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.723322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6c4f-9636-453e-8a33-7840f100d543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.728694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd6c4f-9636-453e-8a33-7840f100d543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:00.728694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd6c4f-9636-453e-8a33-7840f100d543 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:00.736963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1036/2079] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:00.741236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.754554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25108-4f2d-41c6-8fdd-c4a711a9c39d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:00.762636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:00.763414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.764113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.764264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] PATH_INFO: `/v3/projects/dae0c94e56334b4caf3d42f573c94ba8/users/d1c1d1e4a1e64944a219c244e6e36f3b/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.767460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae0c94e56334b4caf3d42f573c94ba8, user_id=d1c1d1e4a1e64944a219c244e6e36f3b, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:00.770337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:00.782026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1044/2080] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:00 2022] PUT /identity/v3/projects/dae0c94e56334b4caf3d42f573c94ba8/users/d1c1d1e4a1e64944a219c244e6e36f3b/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:00.792353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.792726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.793001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:00.822943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] MFA Rules not processed for user `d1c1d1e4a1e64944a219c244e6e36f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:00.824172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:00.846739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:00.896134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1037/2081] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:00.907743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.958890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:00.960271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:00.964979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb429d7-703d-42e2-9f31-bafc13c8cab7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:00.968780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246ea3a-97ce-4c66-8afb-8cda6e1b5ed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:00.969447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246ea3a-97ce-4c66-8afb-8cda6e1b5ed4 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:00.969766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246ea3a-97ce-4c66-8afb-8cda6e1b5ed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:00.970046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246ea3a-97ce-4c66-8afb-8cda6e1b5ed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:01.029254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246ea3a-97ce-4c66-8afb-8cda6e1b5ed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:01.030814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246ea3a-97ce-4c66-8afb-8cda6e1b5ed4 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:01.032608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3e5ac-033a-4af4-9739-b1890fdbc264 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:01.037250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:01.037930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:01.038209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:01.038481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] PATH_INFO: `/v3/users/5fd0f3a5ff4c4dcf81978d55795fcd50` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:01.039613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd0f3a5ff4c4dcf81978d55795fcd50)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:01.040112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1045/2082] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:01.041636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:01.104233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:01.230284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd0f3a5ff4c4dcf81978d55795fcd50'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:01.230965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd0f3a5ff4c4dcf81978d55795fcd50'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:01.307393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd0f3a5ff4c4dcf81978d55795fcd50'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:01.308096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd0f3a5ff4c4dcf81978d55795fcd50'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:01.324754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd0f3a5ff4c4dcf81978d55795fcd50'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:01.341601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1038/2083] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:00 2022] DELETE /identity/v3/users/5fd0f3a5ff4c4dcf81978d55795fcd50 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:01.347223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb883922-66ba-4f38-ba45-d8149aaa8a41 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:01.347506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb883922-66ba-4f38-ba45-d8149aaa8a41 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:01.347892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb883922-66ba-4f38-ba45-d8149aaa8a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:01.369787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-eb883922-66ba-4f38-ba45-d8149aaa8a41 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:01.371286 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eb883922-66ba-4f38-ba45-d8149aaa8a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:01.433291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1046/2084] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:01.443778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:01.445278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:01.485222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:01.506393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec511-cb26-4657-8782-47c6d662719c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:01.510177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:01.510779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:01.511056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:01.511451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:01.521772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:01.523314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:01.532891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1039/2085] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:01.680467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0f0ac-56a8-43d8-8435-4be3f5e49015 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:01.680756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0f0ac-56a8-43d8-8435-4be3f5e49015 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:01.681031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0f0ac-56a8-43d8-8435-4be3f5e49015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:01.703469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-07b0f0ac-56a8-43d8-8435-4be3f5e49015 None None] MFA Rules not processed for user `a8925da59209424fbd6375f056aee752`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:01.704756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-07b0f0ac-56a8-43d8-8435-4be3f5e49015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:01.767013 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1047/2086] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:02.349744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:02.349744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:02.363387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6b39f-4513-4fef-82fd-bdd433d53810 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:02.368088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:02.376568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:02.376568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:02.376568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] PATH_INFO: `/v3/projects/2e0afb6c943047b3a459c7cabfee9c48` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:02.376568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0afb6c943047b3a459c7cabfee9c48)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:02.376568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:02.524777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0afb6c943047b3a459c7cabfee9c48', 'request_id': 'req-643a6a0e-37d9-4c3a-bf88-263828611709'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.573542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0afb6c943047b3a459c7cabfee9c48', 'request_id': 'req-643a6a0e-37d9-4c3a-bf88-263828611709'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.581298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0afb6c943047b3a459c7cabfee9c48', 'request_id': 'req-643a6a0e-37d9-4c3a-bf88-263828611709'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.594855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] The token cache is being invalidate because project 2e0afb6c943047b3a459c7cabfee9c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:02.595310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.596124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.597811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1040/2087] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:02 2022] DELETE /identity/v3/projects/2e0afb6c943047b3a459c7cabfee9c48 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:02.602111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0f0ac-56a8-43d8-8435-4be3f5e49015 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:02.619715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0f0ac-56a8-43d8-8435-4be3f5e49015 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:02.682211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:02.683011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:02.683434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:02.683803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] PATH_INFO: `/v3/users/f4095573a757470cacb496aa64f0f8a9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:02.685059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4095573a757470cacb496aa64f0f8a9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:02.687899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:02.699159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:02.850232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4095573a757470cacb496aa64f0f8a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.850936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4095573a757470cacb496aa64f0f8a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.877703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4095573a757470cacb496aa64f0f8a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.959004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4095573a757470cacb496aa64f0f8a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.959643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4095573a757470cacb496aa64f0f8a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:02.972896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1048/2088] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:02 2022] DELETE /identity/v3/users/f4095573a757470cacb496aa64f0f8a9 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:03.401694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:03.403288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:03.476500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:03.480420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:03.501276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a6a0e-37d9-4c3a-bf88-263828611709 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:03.575605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd250-f74d-4f87-a916-7b1f31df90ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:03.576046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd250-f74d-4f87-a916-7b1f31df90ce service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:03.576386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd250-f74d-4f87-a916-7b1f31df90ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:03.576659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd250-f74d-4f87-a916-7b1f31df90ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:03.579193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01efd250-f74d-4f87-a916-7b1f31df90ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:03.602688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:03.607718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1122f9-d796-409e-aca8-36130fc276d5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:03.612762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:03.613481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:03.613858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:03.614208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] PATH_INFO: `/v3/projects/c9c25dee560e4ee39e8b09f4dd451f92` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:03.615472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c25dee560e4ee39e8b09f4dd451f92)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:03.617724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:03.647089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd250-f74d-4f87-a916-7b1f31df90ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:03.648548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd250-f74d-4f87-a916-7b1f31df90ce service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:03.658241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1041/2089] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:03.764749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c25dee560e4ee39e8b09f4dd451f92', 'request_id': 'req-83de97da-05e1-4189-be97-4a8acdf64dd8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:03.772201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c25dee560e4ee39e8b09f4dd451f92', 'request_id': 'req-83de97da-05e1-4189-be97-4a8acdf64dd8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:03.778557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c25dee560e4ee39e8b09f4dd451f92', 'request_id': 'req-83de97da-05e1-4189-be97-4a8acdf64dd8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:03.873426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] The token cache is being invalidate because project c9c25dee560e4ee39e8b09f4dd451f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:03.873691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:03.875094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83de97da-05e1-4189-be97-4a8acdf64dd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:03.880627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1049/2090] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:03 2022] DELETE /identity/v3/projects/c9c25dee560e4ee39e8b09f4dd451f92 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:03.882591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919afe1e-6345-4296-9d4b-ed9caa5dcc6b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:03.883927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919afe1e-6345-4296-9d4b-ed9caa5dcc6b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:03.884427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919afe1e-6345-4296-9d4b-ed9caa5dcc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:03.909051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-919afe1e-6345-4296-9d4b-ed9caa5dcc6b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:03.910472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-919afe1e-6345-4296-9d4b-ed9caa5dcc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:03.984539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1042/2091] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:03.992510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c626449-1296-428a-92b4-173e495cac93 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:03.992510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c626449-1296-428a-92b4-173e495cac93 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:03.993034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c626449-1296-428a-92b4-173e495cac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.017486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8c626449-1296-428a-92b4-173e495cac93 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:04.018914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8c626449-1296-428a-92b4-173e495cac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:04.027633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1050/2092] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:04.032301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919afe1e-6345-4296-9d4b-ed9caa5dcc6b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.042194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919afe1e-6345-4296-9d4b-ed9caa5dcc6b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:04.048144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:04.049370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.049691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.049971 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.050619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:04.050921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:04.052494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:04.076799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c626449-1296-428a-92b4-173e495cac93 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.077726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1043/2093] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:15:04.087224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c626449-1296-428a-92b4-173e495cac93 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:04.092474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:04.093303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.093686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.094163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.094925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:04.096940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:04.117043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:04.245339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1051/2094] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:04 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:04.246498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.257169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf0609-0e78-4473-bef5-950a7b42557f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:04.317741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:04.318354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.318638 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.319074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.319592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:04.321503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:04.411301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1044/2095] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:04 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:04.415477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.426919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e39e8-c857-479e-930e-20d0dcaff033 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:04.431515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:04.432108 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.432388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.432662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.433389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:04.435011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:04.445668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1052/2096] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:04.449632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.460892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9f38-5db5-4d20-b5c1-3d243bdc3b7e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:04.465611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaa8f1-9db0-41cd-9782-2c7edb8d2018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:04.466222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaa8f1-9db0-41cd-9782-2c7edb8d2018 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.466488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaa8f1-9db0-41cd-9782-2c7edb8d2018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.466756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaa8f1-9db0-41cd-9782-2c7edb8d2018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.467513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eaa8f1-9db0-41cd-9782-2c7edb8d2018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:04.469705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eaa8f1-9db0-41cd-9782-2c7edb8d2018 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:04.494536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1045/2097] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:04.503403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.514050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49845f-7225-4a3e-8a68-92a90a06f3dd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:04.522825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:04.525830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.526136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.526412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] PATH_INFO: `/v3/projects/703c56d209d54c15806c487ada7bd672/users/9d6f3b3e46f842fc9aabe0a06199767c/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.542367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703c56d209d54c15806c487ada7bd672, user_id=9d6f3b3e46f842fc9aabe0a06199767c, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:04.546663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:04.623874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1053/2098] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:04 2022] PUT /identity/v3/projects/703c56d209d54c15806c487ada7bd672/users/9d6f3b3e46f842fc9aabe0a06199767c/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:04.635454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce78f-c6c2-41bd-9ee3-1f233ee3e543 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.635830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce78f-c6c2-41bd-9ee3-1f233ee3e543 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.636182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce78f-c6c2-41bd-9ee3-1f233ee3e543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.685639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0d3ce78f-c6c2-41bd-9ee3-1f233ee3e543 None None] MFA Rules not processed for user `9d6f3b3e46f842fc9aabe0a06199767c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:04.687928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0d3ce78f-c6c2-41bd-9ee3-1f233ee3e543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:04.707230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:04.772441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1046/2099] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:04.776891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.843119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11556b4-8e5d-485d-b200-a423b1cd2442 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:04.847943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:04.848650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:04.848947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:04.849283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:04.849951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:04.852324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:04.946868 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1054/2100] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:04 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:04.948256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ce78f-c6c2-41bd-9ee3-1f233ee3e543 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:04.959381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ce78f-c6c2-41bd-9ee3-1f233ee3e543 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.017867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.018482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.018766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.019334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.019987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.021952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.149922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1047/2101] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:04 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:05.150048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.156400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966779e-81b5-4f5d-971e-efb9e94ac1fd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.161361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.162133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.162302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.162559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.163203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.164836 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.182578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1055/2102] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:05.188213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.200041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.203491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:05.207013 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcfc28-6389-4259-8e6a-44daee90d006 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.214814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.215507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.215884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.216333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.217285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.220441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.235660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1048/2103] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:05.236850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.248318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3d5fa-251d-4aa1-9446-15c4986c1768 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.258185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.258922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.259289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.259634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] PATH_INFO: `/v3/projects/6013094ededb4100ae451ad5b80cbf13/users/5344aa60a63e459395a50d7abc67a1b0/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.273013 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f71ab-239f-4758-8e1e-e189302ccb8f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.277338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf0c3d-081f-4ae1-a178-27f6b18d8290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.277969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf0c3d-081f-4ae1-a178-27f6b18d8290 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.278247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf0c3d-081f-4ae1-a178-27f6b18d8290 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.278745 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf0c3d-081f-4ae1-a178-27f6b18d8290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.281682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0bdf0c3d-081f-4ae1-a178-27f6b18d8290 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:05.291650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6013094ededb4100ae451ad5b80cbf13, user_id=5344aa60a63e459395a50d7abc67a1b0, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.291650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.339174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:05.363524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf0c3d-081f-4ae1-a178-27f6b18d8290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.365290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf0c3d-081f-4ae1-a178-27f6b18d8290 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.377549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1056/2104] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:15:05.384841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1049/2105] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:05 2022] PUT /identity/v3/projects/6013094ededb4100ae451ad5b80cbf13/users/5344aa60a63e459395a50d7abc67a1b0/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:05.396170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793c839-73ee-4e48-8059-d1414bb096fc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.396459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793c839-73ee-4e48-8059-d1414bb096fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.396736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793c839-73ee-4e48-8059-d1414bb096fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.425827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6793c839-73ee-4e48-8059-d1414bb096fc None None] MFA Rules not processed for user `5344aa60a63e459395a50d7abc67a1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:05.427027 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6793c839-73ee-4e48-8059-d1414bb096fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:05.491775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1057/2106] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:05.496309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.558914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4741-b998-437f-a355-1a76f9280ad9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.564838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.565455 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.565762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.566039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.567011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.568922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.588182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:05.720569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1050/2107] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:05 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:05.724631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793c839-73ee-4e48-8059-d1414bb096fc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.735491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793c839-73ee-4e48-8059-d1414bb096fc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.798014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.798708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.798994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.799273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.799943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.801918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.892034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1058/2108] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:05 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:05.893258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.903964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5bedf-26fd-4d08-b524-34db1f7df1a6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.908865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.909469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.909774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.910042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.910677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.912389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:05.923364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1051/2109] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:05.927641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:05.938147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1bd6e-9659-44d8-a018-d77d2253efc3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:05.942703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:05.943354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:05.943632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:05.947937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] PATH_INFO: `/v3/projects/bfd97d278cf540cfaa5124fdbcd5cf95/users/926c0bfc77a44720b9cfc68c4a6a18bf/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:05.965826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd97d278cf540cfaa5124fdbcd5cf95, user_id=926c0bfc77a44720b9cfc68c4a6a18bf, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:05.968717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:06.035579 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1059/2110] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:05 2022] PUT /identity/v3/projects/bfd97d278cf540cfaa5124fdbcd5cf95/users/926c0bfc77a44720b9cfc68c4a6a18bf/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:06.039795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:06.050691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33324764-49c3-440a-a95a-058110c71882 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:06.114264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee6add-8992-4032-80ac-da21e179453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:06.114914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee6add-8992-4032-80ac-da21e179453e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:06.115245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee6add-8992-4032-80ac-da21e179453e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:06.115526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee6add-8992-4032-80ac-da21e179453e admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:06.116164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee6add-8992-4032-80ac-da21e179453e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:06.117887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee6add-8992-4032-80ac-da21e179453e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:06.129373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1052/2111] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:06.133913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:06.145907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01db823-8a22-4e09-83cc-8a2491d2b1f9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:06.151793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:06.152551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:06.152833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:06.153106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] PATH_INFO: `/v3/projects/bfd97d278cf540cfaa5124fdbcd5cf95/users/926c0bfc77a44720b9cfc68c4a6a18bf/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:06.155077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd97d278cf540cfaa5124fdbcd5cf95, user_id=926c0bfc77a44720b9cfc68c4a6a18bf, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:06.157958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:06.168499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1060/2112] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:06 2022] PUT /identity/v3/projects/bfd97d278cf540cfaa5124fdbcd5cf95/users/926c0bfc77a44720b9cfc68c4a6a18bf/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:06.180254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:06.180506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:06.180893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:06.212103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] MFA Rules not processed for user `926c0bfc77a44720b9cfc68c4a6a18bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:06.213319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:06.283313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1053/2113] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:06.294988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:06.325153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:06.360555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b772147-8d8d-4110-9c95-919dfb33eab6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:06.364442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8412db-ea3e-4229-9287-3eb6bdb8cac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:06.365057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8412db-ea3e-4229-9287-3eb6bdb8cac9 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:06.365364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8412db-ea3e-4229-9287-3eb6bdb8cac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:06.365677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8412db-ea3e-4229-9287-3eb6bdb8cac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:06.375830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8412db-ea3e-4229-9287-3eb6bdb8cac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:06.377335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8412db-ea3e-4229-9287-3eb6bdb8cac9 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:06.391317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1061/2114] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:06.509812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:06.513675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:06.526837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1d12f-60ba-4ac9-9364-94639f1f090f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:06.530365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aa669-1012-4c48-a9a4-9c9a926ec7ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:06.530974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aa669-1012-4c48-a9a4-9c9a926ec7ca service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:06.531255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aa669-1012-4c48-a9a4-9c9a926ec7ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:06.531531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aa669-1012-4c48-a9a4-9c9a926ec7ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:06.590313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aa669-1012-4c48-a9a4-9c9a926ec7ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:06.591772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aa669-1012-4c48-a9a4-9c9a926ec7ca service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:06.601799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1054/2115] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:11.205144 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796d4f8-5392-4ccf-b10a-c4f0bc5a83e1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:11.206109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796d4f8-5392-4ccf-b10a-c4f0bc5a83e1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:11.206750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796d4f8-5392-4ccf-b10a-c4f0bc5a83e1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:11.208595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1062/2116] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:15:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:15:11.222541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3763d-d9e5-4570-8a8a-9f01f2ad0208 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:11.223096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3763d-d9e5-4570-8a8a-9f01f2ad0208 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:11.223096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3763d-d9e5-4570-8a8a-9f01f2ad0208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:11.231917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:11.244878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-48c3763d-d9e5-4570-8a8a-9f01f2ad0208 None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:11.246361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-48c3763d-d9e5-4570-8a8a-9f01f2ad0208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:11.318936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1055/2117] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:11.331431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e15b-1883-4b69-aacd-ab541dd9aa6c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:11.331720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e15b-1883-4b69-aacd-ab541dd9aa6c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:11.331998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e15b-1883-4b69-aacd-ab541dd9aa6c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:11.333499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1063/2118] 10.222.0.39 () {54 vars in 961 bytes} [Sun Sep 4 16:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:15:11.341699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e64605-8201-4d82-b442-38d42b501c93 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:11.341699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e64605-8201-4d82-b442-38d42b501c93 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:11.341699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e64605-8201-4d82-b442-38d42b501c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:11.347633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e15b-1883-4b69-aacd-ab541dd9aa6c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:11.349398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4ae6e15b-1883-4b69-aacd-ab541dd9aa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:11.367278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-40e64605-8201-4d82-b442-38d42b501c93 None None] MFA Rules not processed for user `07e04ce012c849baa403dda571f7454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:11.368847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-40e64605-8201-4d82-b442-38d42b501c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:11.379995 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1056/2119] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Sep 4 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:11.388932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e64605-8201-4d82-b442-38d42b501c93 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:11.403641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e64605-8201-4d82-b442-38d42b501c93 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:11.415684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:11.416316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:11.417203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:11.417496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:11.429662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:11.432563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:11.432963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e15b-1883-4b69-aacd-ab541dd9aa6c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:11.438165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:11.438808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:11.439272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:11.439555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] PATH_INFO: `/v3/users/f1af03ad31a34f319763a6b5451ae26e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:11.440725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1af03ad31a34f319763a6b5451ae26e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:11.443148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:11.449741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1057/2120] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:15:11.690005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1af03ad31a34f319763a6b5451ae26e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:11.690673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1af03ad31a34f319763a6b5451ae26e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:11.705026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1af03ad31a34f319763a6b5451ae26e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:11.782997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1af03ad31a34f319763a6b5451ae26e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:11.783633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1eef0c01-472c-41cf-b2d6-d5dd0ab6f9d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1af03ad31a34f319763a6b5451ae26e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:11.796949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1064/2121] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:11 2022] DELETE /identity/v3/users/f1af03ad31a34f319763a6b5451ae26e => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:12.057182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:12.071313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ef26-4b4a-4164-b3fe-d997a796a6ab service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:12.130737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:12.131375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.131659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.131936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] PATH_INFO: `/v3/projects/17338e3b6b204595a43fddc1c00ed26b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.133060 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17338e3b6b204595a43fddc1c00ed26b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:12.134988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:12.286401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17338e3b6b204595a43fddc1c00ed26b', 'request_id': 'req-73519362-287d-4f48-90b3-acbd227675f3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:12.365998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17338e3b6b204595a43fddc1c00ed26b', 'request_id': 'req-73519362-287d-4f48-90b3-acbd227675f3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:12.372068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17338e3b6b204595a43fddc1c00ed26b', 'request_id': 'req-73519362-287d-4f48-90b3-acbd227675f3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:12.383768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] The token cache is being invalidate because project 17338e3b6b204595a43fddc1c00ed26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:12.384057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:12.384680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73519362-287d-4f48-90b3-acbd227675f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:12.386186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1058/2122] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:12 2022] DELETE /identity/v3/projects/17338e3b6b204595a43fddc1c00ed26b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:12.391985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62003d43-1c07-4901-9f77-780a231df3a2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.392498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62003d43-1c07-4901-9f77-780a231df3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.392589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62003d43-1c07-4901-9f77-780a231df3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.412214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-62003d43-1c07-4901-9f77-780a231df3a2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:12.413422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-62003d43-1c07-4901-9f77-780a231df3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:12.451293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:12.471604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1065/2123] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:12.479191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56363b6e-ab6b-49d9-975c-31d39683e8a3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.479562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56363b6e-ab6b-49d9-975c-31d39683e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.479846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56363b6e-ab6b-49d9-975c-31d39683e8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.500437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-56363b6e-ab6b-49d9-975c-31d39683e8a3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:12.501735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-56363b6e-ab6b-49d9-975c-31d39683e8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:12.515734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1059/2124] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:15:12.516962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62003d43-1c07-4901-9f77-780a231df3a2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:12.527444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62003d43-1c07-4901-9f77-780a231df3a2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:12.533111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:12.533742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.534028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.534309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.534951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:12.535255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:12.536983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:12.549667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1066/2125] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:15:12.554133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56363b6e-ab6b-49d9-975c-31d39683e8a3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:12.559768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:12.563667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56363b6e-ab6b-49d9-975c-31d39683e8a3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:12.568431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:12.569023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.569299 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.569586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.570668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:12.572632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:12.681716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1060/2126] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:12 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:12.686192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:12.697035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4df36-c160-4c4c-823e-6a7fe25c32bc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:12.759268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:12.759894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.760179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.760459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.761207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:12.763632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:12.855211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1067/2127] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:12 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:12.856430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:12.867538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2c976-efdc-4d54-91ce-72d8661f83fa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:12.873946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:12.878602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.878602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.878602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.878602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:12.878602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:12.894362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1061/2128] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:12.895087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:12.905992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6b046-af6b-424d-9926-3471f69324d2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:12.910838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b484b-0036-4bd7-8363-e47ca7887662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:12.911428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b484b-0036-4bd7-8363-e47ca7887662 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.911710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b484b-0036-4bd7-8363-e47ca7887662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.911998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b484b-0036-4bd7-8363-e47ca7887662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.912633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b484b-0036-4bd7-8363-e47ca7887662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:12.914305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b484b-0036-4bd7-8363-e47ca7887662 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:12.925365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1068/2129] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:12.929546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:12.939555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fcc5b-947a-4088-a90a-9c30e8bb446b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:12.944300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:12.944937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:12.945211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:12.945485 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] PATH_INFO: `/v3/projects/0df18a4ec765452d983a4ae1e6b479c1/users/19ab134176db48b6ad96aae0589c8e9b/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:12.962901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df18a4ec765452d983a4ae1e6b479c1, user_id=19ab134176db48b6ad96aae0589c8e9b, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:12.971447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:13.046327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1062/2130] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:12 2022] PUT /identity/v3/projects/0df18a4ec765452d983a4ae1e6b479c1/users/19ab134176db48b6ad96aae0589c8e9b/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:13.056605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:13.056897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:13.057171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:13.086763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] MFA Rules not processed for user `19ab134176db48b6ad96aae0589c8e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:13.088031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:13.161850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1069/2131] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:13.173870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:13.175867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:13.243830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252e7e1-24cc-4f22-a9f7-9da851b8b1df admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:13.247540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bf6c2-e30b-47da-a91f-bb6bc9486acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:13.248148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bf6c2-e30b-47da-a91f-bb6bc9486acf service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:13.248425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bf6c2-e30b-47da-a91f-bb6bc9486acf service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:13.248693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bf6c2-e30b-47da-a91f-bb6bc9486acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:13.257974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71bf6c2-e30b-47da-a91f-bb6bc9486acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:13.259386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71bf6c2-e30b-47da-a91f-bb6bc9486acf service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:13.269669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1063/2132] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:13.844311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:13.844311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:13.882356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:13.907187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23cb7-0c9f-4f13-8707-5018636cbe5b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:13.912504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:13.913145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:13.913424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:13.913735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] PATH_INFO: `/v3/users/351ef66ee191480b8d36535bfc55e52f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:13.914870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=351ef66ee191480b8d36535bfc55e52f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:13.917179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:14.151747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351ef66ee191480b8d36535bfc55e52f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:14.161125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351ef66ee191480b8d36535bfc55e52f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:14.175651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351ef66ee191480b8d36535bfc55e52f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:14.246668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351ef66ee191480b8d36535bfc55e52f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:14.247839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351ef66ee191480b8d36535bfc55e52f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:14.264044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1070/2133] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:13 2022] DELETE /identity/v3/users/351ef66ee191480b8d36535bfc55e52f => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:14.270231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:14.270727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:14.271444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:14.304169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:14.305878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:14.369467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1064/2134] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:14.383778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:14.385254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:14.450032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b485ce-4a95-4dbf-b1d6-62053559cb58 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:14.454649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:14.455316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:14.455601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:14.455883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:14.466916 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:14.469053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:14.484785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1071/2135] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:14.939167 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:14.940979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:14.955126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dca6bf-d3f1-4ede-ba00-bc90910dfde6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:14.960727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:14.961599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:14.961972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:14.962339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] PATH_INFO: `/v3/projects/26c1939be17c457a853903373f769562` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:14.971549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26c1939be17c457a853903373f769562)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:14.973578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:15.005380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:15.006903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:15.020737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144311e-0f24-4b4a-bea7-27e9636a77c0 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:15.025341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:15.026019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:15.026305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:15.026613 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] PATH_INFO: `/v3/users/9d6f3b3e46f842fc9aabe0a06199767c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:15.027749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d6f3b3e46f842fc9aabe0a06199767c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:15.029852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:15.092689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:15.092689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26c1939be17c457a853903373f769562', 'request_id': 'req-410f8444-02ac-41a2-a731-677d5097b373'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.152846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26c1939be17c457a853903373f769562', 'request_id': 'req-410f8444-02ac-41a2-a731-677d5097b373'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.159740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26c1939be17c457a853903373f769562', 'request_id': 'req-410f8444-02ac-41a2-a731-677d5097b373'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.171867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] The token cache is being invalidate because project 26c1939be17c457a853903373f769562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:15.172218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.172906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.174575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1065/2136] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:14 2022] DELETE /identity/v3/projects/26c1939be17c457a853903373f769562 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:15.178334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:15.179701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:15.228308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d6f3b3e46f842fc9aabe0a06199767c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.228983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d6f3b3e46f842fc9aabe0a06199767c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.237819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:15.248434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d6f3b3e46f842fc9aabe0a06199767c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.250732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f8444-02ac-41a2-a731-677d5097b373 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:15.255985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:15.256624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:15.256909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:15.257188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] PATH_INFO: `/v3/users/33b6f54f6d10462d9e6323166aafce0b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:15.258378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b6f54f6d10462d9e6323166aafce0b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:15.260319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:15.320492 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d6f3b3e46f842fc9aabe0a06199767c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.322389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c553c5f6-ce36-4292-95be-fbb4b4b48f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d6f3b3e46f842fc9aabe0a06199767c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.336114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1072/2137] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:15 2022] DELETE /identity/v3/users/9d6f3b3e46f842fc9aabe0a06199767c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:15.340320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:15.340606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:15.340878 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:15.364714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:15.366026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:15.425634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1073/2138] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:15:15.435098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:15.437658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:15.482953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b6f54f6d10462d9e6323166aafce0b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.484651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b6f54f6d10462d9e6323166aafce0b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.536721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f78bb-6f14-4c37-bf72-3a9e7a7ff935 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:15.540597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:15.541208 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:15.541549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:15.541824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:15.543644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:15.556185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:15.557726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:15.568719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1074/2139] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:15:15.576336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b6f54f6d10462d9e6323166aafce0b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.576985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b6f54f6d10462d9e6323166aafce0b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.593073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b6f54f6d10462d9e6323166aafce0b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:15.607577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1066/2140] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:15 2022] DELETE /identity/v3/users/33b6f54f6d10462d9e6323166aafce0b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:16.128263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:16.129860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:16.157422 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42bd9f-a088-49a2-9898-0bd18250eb20 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:16.172816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:16.174739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:16.232103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:16.232884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:16.233639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:16.233995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] PATH_INFO: `/v3/projects/703c56d209d54c15806c487ada7bd672` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:16.235361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=703c56d209d54c15806c487ada7bd672)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:16.237357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:16.258777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9fab5-be15-42fb-b0ea-49d0127ed220 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:16.264312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:16.265099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:16.265545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:16.265890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] PATH_INFO: `/v3/projects/2cb8628b6e79489eae93c648ccc7c43c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:16.267091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb8628b6e79489eae93c648ccc7c43c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:16.269086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:16.344922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703c56d209d54c15806c487ada7bd672', 'request_id': 'req-15025de5-95fb-487b-9a04-5df23257bfe6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.350491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703c56d209d54c15806c487ada7bd672', 'request_id': 'req-15025de5-95fb-487b-9a04-5df23257bfe6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.356700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703c56d209d54c15806c487ada7bd672', 'request_id': 'req-15025de5-95fb-487b-9a04-5df23257bfe6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.407265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb8628b6e79489eae93c648ccc7c43c', 'request_id': 'req-74c0435e-c532-4d50-b4c2-9b4a91d82d43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.421834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] The token cache is being invalidate because project 703c56d209d54c15806c487ada7bd672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:16.422126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.422900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.424372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1075/2141] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:16 2022] DELETE /identity/v3/projects/703c56d209d54c15806c487ada7bd672 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:16.428505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:16.429893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:16.496101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15025de5-95fb-487b-9a04-5df23257bfe6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:16.613327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:16.613327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:16.613327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:16.613327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] PATH_INFO: `/v3/users/5344aa60a63e459395a50d7abc67a1b0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:16.614534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5344aa60a63e459395a50d7abc67a1b0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:16.614534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:16.646155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb8628b6e79489eae93c648ccc7c43c', 'request_id': 'req-74c0435e-c532-4d50-b4c2-9b4a91d82d43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.681913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb8628b6e79489eae93c648ccc7c43c', 'request_id': 'req-74c0435e-c532-4d50-b4c2-9b4a91d82d43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.681913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] The token cache is being invalidate because project 2cb8628b6e79489eae93c648ccc7c43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:16.681913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.681913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.681913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1067/2142] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:16 2022] DELETE /identity/v3/projects/2cb8628b6e79489eae93c648ccc7c43c => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:16.682270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:16.682270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:16.698869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0435e-c532-4d50-b4c2-9b4a91d82d43 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:16.705976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:16.706718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:16.706899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:16.707476 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] PATH_INFO: `/v3/users/d1c1d1e4a1e64944a219c244e6e36f3b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:16.708599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c1d1e4a1e64944a219c244e6e36f3b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:16.710670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:16.852275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:16.867031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5344aa60a63e459395a50d7abc67a1b0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.867691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5344aa60a63e459395a50d7abc67a1b0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.890434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5344aa60a63e459395a50d7abc67a1b0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.950393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c1d1e4a1e64944a219c244e6e36f3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.951054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c1d1e4a1e64944a219c244e6e36f3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.964752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5344aa60a63e459395a50d7abc67a1b0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.965991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5344aa60a63e459395a50d7abc67a1b0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:16.987173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1076/2143] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:16 2022] DELETE /identity/v3/users/5344aa60a63e459395a50d7abc67a1b0 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:17.079588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c1d1e4a1e64944a219c244e6e36f3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.080699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c1d1e4a1e64944a219c244e6e36f3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.093816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c1d1e4a1e64944a219c244e6e36f3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.106311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1068/2144] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:16 2022] DELETE /identity/v3/users/d1c1d1e4a1e64944a219c244e6e36f3b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:17.136245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:17.486280 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:17.490050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:17.554614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0a8c-144e-4e8b-802a-0391bd9c83f8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:17.572505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:17.574623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:17.623552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:17.625060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:17.625410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:17.625903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] PATH_INFO: `/v3/projects/6013094ededb4100ae451ad5b80cbf13` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:17.627252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6013094ededb4100ae451ad5b80cbf13)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:17.629339 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:17.666452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5074b-dbb3-4797-866d-18d9c97e6952 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:17.671591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:17.672298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:17.672574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:17.672920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] PATH_INFO: `/v3/projects/dae0c94e56334b4caf3d42f573c94ba8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:17.674133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae0c94e56334b4caf3d42f573c94ba8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:17.676021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:17.779520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6013094ededb4100ae451ad5b80cbf13', 'request_id': 'req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.792287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6013094ededb4100ae451ad5b80cbf13', 'request_id': 'req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.800461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6013094ededb4100ae451ad5b80cbf13', 'request_id': 'req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.818793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae0c94e56334b4caf3d42f573c94ba8', 'request_id': 'req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.895648 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] The token cache is being invalidate because project 6013094ededb4100ae451ad5b80cbf13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:17.895942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.896559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.901695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1077/2145] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:17 2022] DELETE /identity/v3/projects/6013094ededb4100ae451ad5b80cbf13 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:17.902778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:17.904113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:17.970363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae0c94e56334b4caf3d42f573c94ba8', 'request_id': 'req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.975355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123bb0-d1d6-4b72-b4b7-743caca79ed3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:17.980271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae0c94e56334b4caf3d42f573c94ba8', 'request_id': 'req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.989989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] The token cache is being invalidate because project dae0c94e56334b4caf3d42f573c94ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:17.990709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.991342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e5687839-2b73-4fc5-9c45-6ff1eb2b8430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:17.993288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1069/2146] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:17 2022] DELETE /identity/v3/projects/dae0c94e56334b4caf3d42f573c94ba8 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:17.998024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb04e-e847-4959-895c-87cac99aa6f4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:17.998310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb04e-e847-4959-895c-87cac99aa6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:17.998800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb04e-e847-4959-895c-87cac99aa6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.022328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-733cb04e-e847-4959-895c-87cac99aa6f4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:18.023539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-733cb04e-e847-4959-895c-87cac99aa6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:18.042367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.043100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.043497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.043845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] PATH_INFO: `/v3/users/926c0bfc77a44720b9cfc68c4a6a18bf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.045074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=926c0bfc77a44720b9cfc68c4a6a18bf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.047782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.086518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1070/2147] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:15:18.094768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed80180-6d13-4f49-897f-e4e4e2300b94 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.095057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed80180-6d13-4f49-897f-e4e4e2300b94 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.095342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed80180-6d13-4f49-897f-e4e4e2300b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.115808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fed80180-6d13-4f49-897f-e4e4e2300b94 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:18.117722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fed80180-6d13-4f49-897f-e4e4e2300b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:18.127220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1071/2148] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:18.131781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed80180-6d13-4f49-897f-e4e4e2300b94 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.141378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed80180-6d13-4f49-897f-e4e4e2300b94 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.146553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.147156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.147434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.147709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.148660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:18.148963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.150591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.162604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1072/2149] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:15:18.166967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.177998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79731a-00dc-4d16-9bf2-b59a48e446d9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.182821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.183411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.183696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.184006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.184635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.187277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.209776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926c0bfc77a44720b9cfc68c4a6a18bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:18.210448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926c0bfc77a44720b9cfc68c4a6a18bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:18.222728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926c0bfc77a44720b9cfc68c4a6a18bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:18.312511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926c0bfc77a44720b9cfc68c4a6a18bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:18.313164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926c0bfc77a44720b9cfc68c4a6a18bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:18.324293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1073/2150] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:18 2022] POST /identity/v3/projects => generated 399 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:18.325409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.330105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1078/2151] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:17 2022] DELETE /identity/v3/users/926c0bfc77a44720b9cfc68c4a6a18bf => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:18.330265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:18.395597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6acfc-9644-48d8-af13-ae1d40371e42 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.451855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.456014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.456014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.456014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.456014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.456400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.544775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1074/2152] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:18 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:18.550847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.562233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63ae95-d557-48a5-b440-8d0b78b81d51 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.567014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.567622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.567905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.568178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.568823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.570456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.585301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1079/2153] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:18.586241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.597289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f8b6b-e321-4601-82ab-b7eaa37ffabd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.602316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd87d22-5781-46c9-99a4-49af19757dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.603054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd87d22-5781-46c9-99a4-49af19757dde admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.603336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd87d22-5781-46c9-99a4-49af19757dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.603641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd87d22-5781-46c9-99a4-49af19757dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.604287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd87d22-5781-46c9-99a4-49af19757dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.605982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd87d22-5781-46c9-99a4-49af19757dde admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.617342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1075/2154] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:18.621720 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.633148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84421e3d-8911-4514-be5a-242f98846daf admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.638543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.638905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.639182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.639456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] PATH_INFO: `/v3/projects/1ea5eb8c0fcb4029b8083026bf20d08c/users/62123744ded44e9c99610a26a0fb2ccf/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.656295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea5eb8c0fcb4029b8083026bf20d08c, user_id=62123744ded44e9c99610a26a0fb2ccf, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.659225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.765712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1080/2155] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:18 2022] PUT /identity/v3/projects/1ea5eb8c0fcb4029b8083026bf20d08c/users/62123744ded44e9c99610a26a0fb2ccf/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:18.776841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.777316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.777409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.794773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.803379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:18.818246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] MFA Rules not processed for user `62123744ded44e9c99610a26a0fb2ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:18.820074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:18.891852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1076/2156] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:18.896323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:18.899414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7669f-f327-4fab-a0e9-83a67471f804 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.903996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:18.907052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:18.907168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.907168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.907168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.907168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] PATH_INFO: `/v3/projects/bfd97d278cf540cfaa5124fdbcd5cf95` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.908308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd97d278cf540cfaa5124fdbcd5cf95)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.910528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.967432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f80d4-19d0-45b7-83ab-5310f354d229 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:18.972084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fee2b7-8c7f-4384-94fa-04d112501abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:18.972696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fee2b7-8c7f-4384-94fa-04d112501abc service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:18.972976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fee2b7-8c7f-4384-94fa-04d112501abc service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:18.973249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fee2b7-8c7f-4384-94fa-04d112501abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:18.982042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fee2b7-8c7f-4384-94fa-04d112501abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:18.984063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fee2b7-8c7f-4384-94fa-04d112501abc service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:18.993150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1077/2157] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:19.039914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd97d278cf540cfaa5124fdbcd5cf95', 'request_id': 'req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:19.046798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd97d278cf540cfaa5124fdbcd5cf95', 'request_id': 'req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:19.053367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd97d278cf540cfaa5124fdbcd5cf95', 'request_id': 'req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:19.130073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] The token cache is being invalidate because project bfd97d278cf540cfaa5124fdbcd5cf95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:19.130073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:19.130073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ceef2e4a-2cad-4b82-8def-1ba8c0eb056f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:19.130946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1081/2158] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:18 2022] DELETE /identity/v3/projects/bfd97d278cf540cfaa5124fdbcd5cf95 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:19.139099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12faafe-822e-4809-8b48-0f6daf0d7277 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.139395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12faafe-822e-4809-8b48-0f6daf0d7277 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.139706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12faafe-822e-4809-8b48-0f6daf0d7277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.174629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e12faafe-822e-4809-8b48-0f6daf0d7277 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:19.175891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e12faafe-822e-4809-8b48-0f6daf0d7277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:19.190312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:19.250020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1078/2159] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:15:19.252267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1443d47-99db-43be-9e96-03ea6620e02a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.252758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1443d47-99db-43be-9e96-03ea6620e02a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.252869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1443d47-99db-43be-9e96-03ea6620e02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.281116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a1443d47-99db-43be-9e96-03ea6620e02a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:19.282497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1443d47-99db-43be-9e96-03ea6620e02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:19.291166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1082/2160] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:19.295433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12faafe-822e-4809-8b48-0f6daf0d7277 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:19.305412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12faafe-822e-4809-8b48-0f6daf0d7277 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:19.310075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:19.310664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.310941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.311204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.312379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:19.312673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:19.314366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:19.326467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1079/2161] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:15:19.331717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1443d47-99db-43be-9e96-03ea6620e02a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:19.341091 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1443d47-99db-43be-9e96-03ea6620e02a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:19.345947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:19.346491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.346768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.347036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.347671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:19.349587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:19.435662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1083/2162] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:19 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:19.446127 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:19.457776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57700a3d-fb23-40b9-800f-26ace0ac6f41 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:19.514328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:19.515452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.515924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.516180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.516820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:19.518748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:19.603153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1080/2163] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:19 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:19.604635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:19.615981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2f9e7-7355-4911-bce0-cfffd3156309 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:19.620587 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:19.621241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.621542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.621837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.622564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:19.625028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:19.635353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1084/2164] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:19.639623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:19.649649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d2d8b-9f5a-4e0f-9c70-667097b75816 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:19.654296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3b433-bb80-4632-8c14-899a64737c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:19.654932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3b433-bb80-4632-8c14-899a64737c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.655270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3b433-bb80-4632-8c14-899a64737c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.655550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3b433-bb80-4632-8c14-899a64737c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.656281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3b433-bb80-4632-8c14-899a64737c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:19.657915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3b433-bb80-4632-8c14-899a64737c2d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:19.668639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1081/2165] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:19.673049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:19.683251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae879d28-679d-405c-9757-0fd0472b61a9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:19.689121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:19.689799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.690086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.690369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] PATH_INFO: `/v3/projects/d611c18ba5954661af7d3c604da49919/users/bbccfac2bf8a418584d07e877bb390f7/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.705428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d611c18ba5954661af7d3c604da49919, user_id=bbccfac2bf8a418584d07e877bb390f7, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:19.708170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:19.802876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1085/2166] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:19 2022] PUT /identity/v3/projects/d611c18ba5954661af7d3c604da49919/users/bbccfac2bf8a418584d07e877bb390f7/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:19.813397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b11697-386e-4df4-83c5-dd5e4348dbd4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.813933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b11697-386e-4df4-83c5-dd5e4348dbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.814051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b11697-386e-4df4-83c5-dd5e4348dbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.842836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-58b11697-386e-4df4-83c5-dd5e4348dbd4 None None] MFA Rules not processed for user `bbccfac2bf8a418584d07e877bb390f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:19.844196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-58b11697-386e-4df4-83c5-dd5e4348dbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:19.915536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1082/2167] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:19.916884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:19.980430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08507b2e-9b83-4da8-b806-f910979d0dec admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:19.986097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:19.986696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:19.986980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:19.987259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:19.987884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:19.989830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:20.100256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1086/2168] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:19 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:20.104000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b11697-386e-4df4-83c5-dd5e4348dbd4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:20.115682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b11697-386e-4df4-83c5-dd5e4348dbd4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:20.167272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:20.175125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:20.175817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:20.176102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:20.176357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:20.177002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:20.178943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:20.273181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1083/2169] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:20 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:20.281014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:20.291636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ed287-a7a1-465d-a406-dfb7ebd58afc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:20.297111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:20.297816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:20.300724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:20.300724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:20.300724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:20.300724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:20.311352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1087/2170] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:20.316125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:20.326875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25441cd4-4106-416a-9353-616c35c6b4ad admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:20.338915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:20.339061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:20.339263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:20.339534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] PATH_INFO: `/v3/projects/4d48485b3dfb460eb68f4c6741a14ae5/users/b5046ca2c9d54f1cb8548901fd6091de/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:20.355764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d48485b3dfb460eb68f4c6741a14ae5, user_id=b5046ca2c9d54f1cb8548901fd6091de, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:20.358648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:20.444832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1084/2171] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:20 2022] PUT /identity/v3/projects/4d48485b3dfb460eb68f4c6741a14ae5/users/b5046ca2c9d54f1cb8548901fd6091de/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:20.448823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:20.459487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3c90-8baa-49dc-ac12-c3f9dfee0e80 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:20.514325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a537a-4d91-4baa-9242-7a64718ab4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:20.515604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a537a-4d91-4baa-9242-7a64718ab4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:20.515882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a537a-4d91-4baa-9242-7a64718ab4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:20.516155 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a537a-4d91-4baa-9242-7a64718ab4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:20.516792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a537a-4d91-4baa-9242-7a64718ab4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:20.518531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a537a-4d91-4baa-9242-7a64718ab4fa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:20.528350 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1088/2172] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:20.533240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:20.543645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998acbb-df26-43ac-858a-38ddb64b5a77 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:20.548410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:20.549924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:20.550308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:20.550587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] PATH_INFO: `/v3/projects/4d48485b3dfb460eb68f4c6741a14ae5/users/b5046ca2c9d54f1cb8548901fd6091de/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:20.553154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d48485b3dfb460eb68f4c6741a14ae5, user_id=b5046ca2c9d54f1cb8548901fd6091de, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:20.555850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:20.567313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1085/2173] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:20 2022] PUT /identity/v3/projects/4d48485b3dfb460eb68f4c6741a14ae5/users/b5046ca2c9d54f1cb8548901fd6091de/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:20.579100 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf30ec8-4b79-4f46-8535-3b69b5d82155 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:20.579398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf30ec8-4b79-4f46-8535-3b69b5d82155 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:20.579683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf30ec8-4b79-4f46-8535-3b69b5d82155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:20.608882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fcf30ec8-4b79-4f46-8535-3b69b5d82155 None None] MFA Rules not processed for user `b5046ca2c9d54f1cb8548901fd6091de`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:20.610174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fcf30ec8-4b79-4f46-8535-3b69b5d82155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:20.678924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1089/2174] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:20.690378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:20.692181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:20.748194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf0367-d28f-4ef2-95f1-9b5ee8b711f8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:20.752482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:20.753084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:20.753369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:20.753682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:20.762779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:20.764230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:20.774799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1086/2175] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:20.789278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:21.000977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf30ec8-4b79-4f46-8535-3b69b5d82155 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:21.025146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf30ec8-4b79-4f46-8535-3b69b5d82155 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:21.031533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ce472-10d5-407d-9f95-45d1aef56a1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:21.032337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ce472-10d5-407d-9f95-45d1aef56a1c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:21.032643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ce472-10d5-407d-9f95-45d1aef56a1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:21.032999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ce472-10d5-407d-9f95-45d1aef56a1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:21.092048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ce472-10d5-407d-9f95-45d1aef56a1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:21.093507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ce472-10d5-407d-9f95-45d1aef56a1c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:21.106332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1090/2176] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:21.812836 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:21.963606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:22.035574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a17eb0-2ee5-49e2-a964-eb2c96a96215 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:22.040295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:22.040923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:22.041203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:22.041502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] PATH_INFO: `/v3/users/bbccfac2bf8a418584d07e877bb390f7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:22.042649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbccfac2bf8a418584d07e877bb390f7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:22.044598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:22.236760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbccfac2bf8a418584d07e877bb390f7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:22.237465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbccfac2bf8a418584d07e877bb390f7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:22.280615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:22.297233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbccfac2bf8a418584d07e877bb390f7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:22.298001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbccfac2bf8a418584d07e877bb390f7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:22.316881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbccfac2bf8a418584d07e877bb390f7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:22.328183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1087/2177] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:21 2022] DELETE /identity/v3/users/bbccfac2bf8a418584d07e877bb390f7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:22.335545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:22.335909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:22.336264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:22.356288 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:22.357487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:22.423411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1091/2178] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:22.433703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:22.435186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:22.500880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d300a-e0ed-4648-bfa8-34f89b91114f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:22.507115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:22.507115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:22.507115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:22.507115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:22.518019 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:22.519805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:22.533989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1088/2179] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:23.006065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:23.037405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:23.039151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:23.063783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e0ac4-9566-4312-b6e0-6440327dfaf2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:23.069064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:23.069879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:23.070230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:23.070574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] PATH_INFO: `/v3/projects/d611c18ba5954661af7d3c604da49919` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:23.071840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d611c18ba5954661af7d3c604da49919)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:23.073881 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:23.220129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd611c18ba5954661af7d3c604da49919', 'request_id': 'req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.225922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd611c18ba5954661af7d3c604da49919', 'request_id': 'req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.233057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd611c18ba5954661af7d3c604da49919', 'request_id': 'req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.332682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] The token cache is being invalidate because project d611c18ba5954661af7d3c604da49919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:23.333381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.333766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.335275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1092/2180] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:23 2022] DELETE /identity/v3/projects/d611c18ba5954661af7d3c604da49919 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:23.339549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:23.349327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6f9bc-c67c-4ca5-9fcc-02852d0686eb service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:23.465922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:23.528634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:23.529289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:23.529591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:23.529862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] PATH_INFO: `/v3/users/b5046ca2c9d54f1cb8548901fd6091de` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:23.531129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5046ca2c9d54f1cb8548901fd6091de)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:23.533038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:23.536868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:23.728233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5046ca2c9d54f1cb8548901fd6091de'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.728906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5046ca2c9d54f1cb8548901fd6091de'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.788515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5046ca2c9d54f1cb8548901fd6091de'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.789140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5046ca2c9d54f1cb8548901fd6091de'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.802482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ddbe346-f2f9-450b-85bb-8b60801d1495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5046ca2c9d54f1cb8548901fd6091de'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:23.814480 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1089/2181] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:23 2022] DELETE /identity/v3/users/b5046ca2c9d54f1cb8548901fd6091de => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:23.875350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:23.949997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:24.239670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:24.241146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:24.254483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3dd40-9928-4301-bc03-6bf0aaecaba1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:24.299955 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:24.314604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:24.315233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:24.315509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:24.315781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] PATH_INFO: `/v3/projects/4d48485b3dfb460eb68f4c6741a14ae5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:24.316995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d48485b3dfb460eb68f4c6741a14ae5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:24.318873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:24.405865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d48485b3dfb460eb68f4c6741a14ae5', 'request_id': 'req-fceebc47-84e0-4686-b163-57196cef02d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:24.411298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d48485b3dfb460eb68f4c6741a14ae5', 'request_id': 'req-fceebc47-84e0-4686-b163-57196cef02d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:24.417299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d48485b3dfb460eb68f4c6741a14ae5', 'request_id': 'req-fceebc47-84e0-4686-b163-57196cef02d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:24.468647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] The token cache is being invalidate because project 4d48485b3dfb460eb68f4c6741a14ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:24.468938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:24.469623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fceebc47-84e0-4686-b163-57196cef02d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:24.471148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1093/2182] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:24 2022] DELETE /identity/v3/projects/4d48485b3dfb460eb68f4c6741a14ae5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:24.476033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b29b9-26e4-4979-bbc8-203a4a09bcd2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:24.476508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b29b9-26e4-4979-bbc8-203a4a09bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:24.476637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b29b9-26e4-4979-bbc8-203a4a09bcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:24.496877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-088b29b9-26e4-4979-bbc8-203a4a09bcd2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:24.498294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-088b29b9-26e4-4979-bbc8-203a4a09bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:24.561006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1090/2183] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:24.571690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afdac2-59f3-4627-951a-1f732601d283 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:24.571975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afdac2-59f3-4627-951a-1f732601d283 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:24.572250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afdac2-59f3-4627-951a-1f732601d283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:24.593368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a8afdac2-59f3-4627-951a-1f732601d283 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:24.594653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a8afdac2-59f3-4627-951a-1f732601d283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:24.602628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1094/2184] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:24.606920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b29b9-26e4-4979-bbc8-203a4a09bcd2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:24.617082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b29b9-26e4-4979-bbc8-203a4a09bcd2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:24.621531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:24.622143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:24.622419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:24.622688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:24.623323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:24.623649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:24.625217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:24.636629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1091/2185] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:15:24.641449 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afdac2-59f3-4627-951a-1f732601d283 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:24.650716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afdac2-59f3-4627-951a-1f732601d283 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:24.656070 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:24.656602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:24.656887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:24.657184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:24.657847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:24.659767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:24.772237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1095/2186] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:24 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:24.776258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:24.786878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace34a-7143-478b-8e38-4159f6fd8407 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:24.843204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:24.843805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:24.844104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:24.844371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:24.845021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:24.846932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:24.963972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1092/2187] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:24 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:24.969199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:24.980133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c7e1d-e8dc-4995-a534-3c1199f42f94 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:24.985222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:24.985851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:24.986132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:24.986405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:24.987097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:24.988698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:25.002481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1096/2188] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:25.003686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:25.013936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb73f1-2e1e-4af2-bfaf-f3b312491686 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:25.019435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767597d-7c97-4e1e-9d05-5ee4a0aec515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:25.020062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767597d-7c97-4e1e-9d05-5ee4a0aec515 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:25.020345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767597d-7c97-4e1e-9d05-5ee4a0aec515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:25.020637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767597d-7c97-4e1e-9d05-5ee4a0aec515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:25.021297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767597d-7c97-4e1e-9d05-5ee4a0aec515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:25.022928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767597d-7c97-4e1e-9d05-5ee4a0aec515 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:25.028087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:25.033816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1093/2189] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:25.040371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:25.051201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64242059-b891-4294-80d7-dcec4841c857 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:25.056009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:25.056744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:25.057022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:25.057297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] PATH_INFO: `/v3/projects/a74fb60110604b798b4362c87a9d1e65/users/60179efd726e44d085da365f369b3110/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:25.072364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74fb60110604b798b4362c87a9d1e65, user_id=60179efd726e44d085da365f369b3110, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:25.075053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:25.171851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1097/2190] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:25 2022] PUT /identity/v3/projects/a74fb60110604b798b4362c87a9d1e65/users/60179efd726e44d085da365f369b3110/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:25.182073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb773-7ff6-4d33-8fe4-2243a0ba7621 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:25.182361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb773-7ff6-4d33-8fe4-2243a0ba7621 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:25.182645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb773-7ff6-4d33-8fe4-2243a0ba7621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:25.211351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5d0bb773-7ff6-4d33-8fe4-2243a0ba7621 None None] MFA Rules not processed for user `60179efd726e44d085da365f369b3110`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:25.212578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5d0bb773-7ff6-4d33-8fe4-2243a0ba7621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:25.278300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1094/2191] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:25.289304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:25.291037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:25.348769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde23d4-44ac-4b16-9817-e0ba13c4e128 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:25.352446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663423c-df87-4cd4-855e-c4d8708736d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:25.353048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663423c-df87-4cd4-855e-c4d8708736d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:25.353330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663423c-df87-4cd4-855e-c4d8708736d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:25.353640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663423c-df87-4cd4-855e-c4d8708736d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:25.361845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663423c-df87-4cd4-855e-c4d8708736d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:25.363282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663423c-df87-4cd4-855e-c4d8708736d6 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:25.371894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1098/2192] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:25.392918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb773-7ff6-4d33-8fe4-2243a0ba7621 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:25.454302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb773-7ff6-4d33-8fe4-2243a0ba7621 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:25.458861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:25.459487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:25.459768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:25.460044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] PATH_INFO: `/v3/users/60179efd726e44d085da365f369b3110` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:25.461174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60179efd726e44d085da365f369b3110)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:25.463114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:25.663796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60179efd726e44d085da365f369b3110'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:25.664468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60179efd726e44d085da365f369b3110'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:25.738767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60179efd726e44d085da365f369b3110'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:25.739421 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60179efd726e44d085da365f369b3110'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:25.758824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60179efd726e44d085da365f369b3110'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:25.773280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1095/2193] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:25 2022] DELETE /identity/v3/users/60179efd726e44d085da365f369b3110 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:25.783473 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:25.783927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:25.784018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:25.804893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:25.809200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:25.815826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:25.865513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1099/2194] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:25.875901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:25.877286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:25.888597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:25.939944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4383c-804b-40d8-9468-4dd06512db0d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:25.944068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f15221-9e2f-4a71-8259-b493cf39c5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:25.944673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f15221-9e2f-4a71-8259-b493cf39c5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:25.944956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f15221-9e2f-4a71-8259-b493cf39c5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:25.945228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f15221-9e2f-4a71-8259-b493cf39c5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:25.955219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f15221-9e2f-4a71-8259-b493cf39c5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:25.956673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f15221-9e2f-4a71-8259-b493cf39c5d8 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:25.966595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1096/2195] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:26.388760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:26.391426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:26.409337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ee858-6995-46ce-b9d4-12d1a042c347 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:26.414752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:26.416623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:26.416907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:26.417524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] PATH_INFO: `/v3/projects/a74fb60110604b798b4362c87a9d1e65` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:26.418746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74fb60110604b798b4362c87a9d1e65)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:26.421604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:26.550523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74fb60110604b798b4362c87a9d1e65', 'request_id': 'req-ad894004-1ca9-4803-a4d8-39a857dd0ba7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:26.556064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74fb60110604b798b4362c87a9d1e65', 'request_id': 'req-ad894004-1ca9-4803-a4d8-39a857dd0ba7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:26.562948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74fb60110604b798b4362c87a9d1e65', 'request_id': 'req-ad894004-1ca9-4803-a4d8-39a857dd0ba7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:26.677456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] The token cache is being invalidate because project a74fb60110604b798b4362c87a9d1e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:26.677787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:26.678462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad894004-1ca9-4803-a4d8-39a857dd0ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:26.679960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1100/2196] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:26 2022] DELETE /identity/v3/projects/a74fb60110604b798b4362c87a9d1e65 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:26.685129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460210c-7827-48d5-802f-3e6923e51851 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:26.685665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460210c-7827-48d5-802f-3e6923e51851 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:26.685775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460210c-7827-48d5-802f-3e6923e51851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:26.705161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5460210c-7827-48d5-802f-3e6923e51851 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:26.706414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5460210c-7827-48d5-802f-3e6923e51851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:26.771565 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1097/2197] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:26.780289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952e3e-08b2-478a-a97b-960247589995 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:26.780603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952e3e-08b2-478a-a97b-960247589995 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:26.780884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952e3e-08b2-478a-a97b-960247589995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:26.801007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-70952e3e-08b2-478a-a97b-960247589995 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:26.802224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70952e3e-08b2-478a-a97b-960247589995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:26.811559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1101/2198] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:26.816228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460210c-7827-48d5-802f-3e6923e51851 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:26.827114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460210c-7827-48d5-802f-3e6923e51851 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:26.832966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:26.833698 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:26.834032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:26.834569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:26.835507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:26.835810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:26.837380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:26.849267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1098/2199] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:15:26.853719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70952e3e-08b2-478a-a97b-960247589995 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:26.863207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70952e3e-08b2-478a-a97b-960247589995 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:26.870553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:26.871151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:26.871431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:26.872787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:26.873659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:26.876167 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:26.985047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1102/2200] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:26 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:26.989659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:27.003496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a09b0-370e-4402-88ab-e31ea7cd8343 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:27.082514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:27.083260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:27.083544 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:27.083819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:27.084632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:27.086705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:27.178398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1099/2201] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:26 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:27.182413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:27.193760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66995ca-38ce-49e5-8493-7ca9af60cd1f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:27.198446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:27.199045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:27.199329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:27.199603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:27.200332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:27.201982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:27.212884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1103/2202] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:27.216863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:27.229111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3b407-b9ad-4269-85ae-41a16319e037 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:27.234460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc0584-13d1-48c3-ad8f-48992a4fd777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:27.235828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc0584-13d1-48c3-ad8f-48992a4fd777 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:27.236111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc0584-13d1-48c3-ad8f-48992a4fd777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:27.236575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc0584-13d1-48c3-ad8f-48992a4fd777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:27.237351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc0584-13d1-48c3-ad8f-48992a4fd777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:27.239284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc0584-13d1-48c3-ad8f-48992a4fd777 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:27.255214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1100/2203] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:27.259324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:27.270405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32ccea-039a-4579-af39-944253d888f9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:27.275107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:27.280092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:27.280092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:27.280092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] PATH_INFO: `/v3/projects/770dbd9183804e7fb0cbb99af75b45f2/users/7796ec614f2545c29c2739d3dfd889f4/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:27.292666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] RBAC: Authorizing `identity:create_grant(project_id=770dbd9183804e7fb0cbb99af75b45f2, user_id=7796ec614f2545c29c2739d3dfd889f4, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:27.295421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:27.372229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1104/2204] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:27 2022] PUT /identity/v3/projects/770dbd9183804e7fb0cbb99af75b45f2/users/7796ec614f2545c29c2739d3dfd889f4/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:27.378341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:27.378632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:27.378931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:27.398425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:27.399768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:27.412276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:27.488450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1101/2205] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:27.498165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:27.500435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:27.549638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:27.579873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09934a-9006-4557-b0c0-3a9b3b58f19f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:27.583729 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe31232-2b2a-4973-a7e3-6ccff15c432c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:27.584400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe31232-2b2a-4973-a7e3-6ccff15c432c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:27.587890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe31232-2b2a-4973-a7e3-6ccff15c432c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:27.587890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe31232-2b2a-4973-a7e3-6ccff15c432c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:27.596114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe31232-2b2a-4973-a7e3-6ccff15c432c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:27.597606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe31232-2b2a-4973-a7e3-6ccff15c432c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:27.609169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1105/2206] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:28.094280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:28.096052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:28.108268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7cc8a-16f7-4634-8cc5-6b0e1af9b1e0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:28.113637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:28.114342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:28.114685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:28.114958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] PATH_INFO: `/v3/users/19ab134176db48b6ad96aae0589c8e9b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:28.116176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ab134176db48b6ad96aae0589c8e9b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:28.118206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:28.344011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ab134176db48b6ad96aae0589c8e9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.344862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ab134176db48b6ad96aae0589c8e9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.394101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ab134176db48b6ad96aae0589c8e9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.394732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ab134176db48b6ad96aae0589c8e9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.407717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ab134176db48b6ad96aae0589c8e9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.423121 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1102/2207] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:28 2022] DELETE /identity/v3/users/19ab134176db48b6ad96aae0589c8e9b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:28.428459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:28.428967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:28.429119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:28.451646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:28.452886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:28.519499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1106/2208] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:28.530446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:28.532766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:28.568464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:28.569977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:28.583097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616739d-603a-41d7-b069-201b68db82d0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:28.588092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:28.588730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:28.589014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:28.589295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] PATH_INFO: `/v3/users/5ce568ea7abb4674ab38463a36c7c3e8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:28.590538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce568ea7abb4674ab38463a36c7c3e8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:28.592795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:28.595549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb7695-6516-48d1-b82f-118846987bcd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:28.602177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5311fb-4b35-4680-bdea-9efdee0330fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:28.603250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5311fb-4b35-4680-bdea-9efdee0330fa service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:28.603250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5311fb-4b35-4680-bdea-9efdee0330fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:28.603484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5311fb-4b35-4680-bdea-9efdee0330fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:28.613932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5311fb-4b35-4680-bdea-9efdee0330fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:28.616123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5311fb-4b35-4680-bdea-9efdee0330fa service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:28.626805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1103/2209] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:28.843601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce568ea7abb4674ab38463a36c7c3e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.845152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce568ea7abb4674ab38463a36c7c3e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.861272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce568ea7abb4674ab38463a36c7c3e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.966306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce568ea7abb4674ab38463a36c7c3e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.966946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce568ea7abb4674ab38463a36c7c3e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:28.988808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1107/2210] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:28 2022] DELETE /identity/v3/users/5ce568ea7abb4674ab38463a36c7c3e8 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:28.994908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:28.994908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:28.996900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.034240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:29.036040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:29.092330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:29.149889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1104/2211] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:15:29.161322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:29.164066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:29.224994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:29.228964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742cb89-d71c-4798-aa94-e419027c6fc3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:29.233175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:29.233878 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:29.234062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:29.234336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.245666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:29.249002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:29.258697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1108/2212] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:15:29.288289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:29.289931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:29.310313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48f75d-341e-49ca-a2d8-95177eceb75c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:29.328392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:29.329751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:29.333608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:29.333608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] PATH_INFO: `/v3/projects/0df18a4ec765452d983a4ae1e6b479c1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.333608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df18a4ec765452d983a4ae1e6b479c1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:29.334339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:29.526496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:29.528203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:29.534042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df18a4ec765452d983a4ae1e6b479c1', 'request_id': 'req-07af55ce-94ca-49a3-9af5-37566c09cc1f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:29.545516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0ebba-17c1-4fe5-a992-da0cea06c130 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:29.630084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:29.631749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:29.631749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:29.631749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] PATH_INFO: `/v3/users/62123744ded44e9c99610a26a0fb2ccf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.633058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=62123744ded44e9c99610a26a0fb2ccf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:29.635123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:29.652445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df18a4ec765452d983a4ae1e6b479c1', 'request_id': 'req-07af55ce-94ca-49a3-9af5-37566c09cc1f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:29.661322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df18a4ec765452d983a4ae1e6b479c1', 'request_id': 'req-07af55ce-94ca-49a3-9af5-37566c09cc1f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:29.675467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] The token cache is being invalidate because project 0df18a4ec765452d983a4ae1e6b479c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:29.675762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:29.676978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-07af55ce-94ca-49a3-9af5-37566c09cc1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:29.678889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1105/2213] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:29 2022] DELETE /identity/v3/projects/0df18a4ec765452d983a4ae1e6b479c1 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:29.694895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a206f-e138-443c-99c5-0349051e50a3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:29.695818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a206f-e138-443c-99c5-0349051e50a3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:29.696105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a206f-e138-443c-99c5-0349051e50a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.726240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5f3a206f-e138-443c-99c5-0349051e50a3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:29.727557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5f3a206f-e138-443c-99c5-0349051e50a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:29.737086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1106/2214] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:29.753100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3eb8d-5f55-406f-a20a-c6cf1fc40e07 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:29.753406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3eb8d-5f55-406f-a20a-c6cf1fc40e07 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:29.754294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3eb8d-5f55-406f-a20a-c6cf1fc40e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.803310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e2e3eb8d-5f55-406f-a20a-c6cf1fc40e07 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:29.804725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2e3eb8d-5f55-406f-a20a-c6cf1fc40e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:29.823875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1107/2215] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:29.825259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3eb8d-5f55-406f-a20a-c6cf1fc40e07 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:29.841880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3eb8d-5f55-406f-a20a-c6cf1fc40e07 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:29.848424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:29.849549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:29.850259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:29.850779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.851714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:29.852249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:29.854791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:29.868925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1108/2216] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:15:29.873331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:29.887747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2250e-f248-402e-8931-71e1f65ef468 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:29.894644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:29.895305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:29.895575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:29.895820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:29.896475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:29.898987 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:29.906906 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62123744ded44e9c99610a26a0fb2ccf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:29.907719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62123744ded44e9c99610a26a0fb2ccf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:29.925506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62123744ded44e9c99610a26a0fb2ccf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.000119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62123744ded44e9c99610a26a0fb2ccf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.001792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e685bd93-9549-4d1c-8746-7a4ef7eef28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62123744ded44e9c99610a26a0fb2ccf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.011955 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1109/2217] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:29 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:30.017355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.019732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.022414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1109/2218] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:15:29 2022] DELETE /identity/v3/users/62123744ded44e9c99610a26a0fb2ccf => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:30.026849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.027360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.027885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.066891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:30.070570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.111983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadfe60-f038-4827-9172-ba28493e74ed admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.133951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:30.175580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1110/2219] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:15:30.175580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.175580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.200490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2d164-b26e-4573-a8cb-765239cd1036 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.202179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.202805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.203130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.203367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.204732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.206912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.208327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.209159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.212195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.212195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] PATH_INFO: `/v3/projects/3ba3f2b6ecf540759b706043dde7c248` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.212195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ba3f2b6ecf540759b706043dde7c248)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.214132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.226805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:30.332433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1110/2220] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:30 2022] POST /identity/v3/users => generated 286 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:30.333881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.334775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.386570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba3f2b6ecf540759b706043dde7c248', 'request_id': 'req-4b167ba4-32f8-4672-a703-07a1e5c0567e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.398533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba3f2b6ecf540759b706043dde7c248', 'request_id': 'req-4b167ba4-32f8-4672-a703-07a1e5c0567e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.408082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba3f2b6ecf540759b706043dde7c248', 'request_id': 'req-4b167ba4-32f8-4672-a703-07a1e5c0567e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.440416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bbc30-81bb-4f49-921a-605488d6d28d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.480198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] The token cache is being invalidate because project 3ba3f2b6ecf540759b706043dde7c248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:30.480677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.481364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:30.483014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1111/2221] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:30 2022] DELETE /identity/v3/projects/3ba3f2b6ecf540759b706043dde7c248 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:30.483913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.487312 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.522250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.523138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.523695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.524180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.526645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.595604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b167ba4-32f8-4672-a703-07a1e5c0567e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.602659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7051e-5dd2-49b8-9e28-8ef3f80d9f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.603526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7051e-5dd2-49b8-9e28-8ef3f80d9f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.603887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7051e-5dd2-49b8-9e28-8ef3f80d9f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.604506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7051e-5dd2-49b8-9e28-8ef3f80d9f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.605627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7051e-5dd2-49b8-9e28-8ef3f80d9f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.607272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7051e-5dd2-49b8-9e28-8ef3f80d9f5b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.622756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.624245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.634808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1112/2222] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:30.635857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9707bc-b82d-4042-808f-3a9daea097ec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.636148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9707bc-b82d-4042-808f-3a9daea097ec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.636588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9707bc-b82d-4042-808f-3a9daea097ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.642739 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1111/2223] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:30.643628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.654501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42e65-bea0-49ba-845b-ae488f481bd2 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.660481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.661101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.661411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.661743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.662557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.664255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.665234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ae9707bc-b82d-4042-808f-3a9daea097ec None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:30.667775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ae9707bc-b82d-4042-808f-3a9daea097ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.673373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:30.680576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1112/2224] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:30.683058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1113/2225] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:30.685016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.696295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06978f-c96f-4fab-9bca-d1b23e83c84b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.696642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06978f-c96f-4fab-9bca-d1b23e83c84b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.696864 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06978f-c96f-4fab-9bca-d1b23e83c84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.729958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d777b93-f7b0-4f52-a510-17fba110d695 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.741654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bf06978f-c96f-4fab-9bca-d1b23e83c84b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:30.741654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bf06978f-c96f-4fab-9bca-d1b23e83c84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:30.748478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e42e6a-9b5a-460c-bfe3-e3ae34c1645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.749219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e42e6a-9b5a-460c-bfe3-e3ae34c1645d admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.749511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e42e6a-9b5a-460c-bfe3-e3ae34c1645d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.749811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e42e6a-9b5a-460c-bfe3-e3ae34c1645d admin admin] PATH_INFO: `/v3/projects/dafff73d9941412fae6e3c8e92591c9f/users/0794c6aadcef4125828554fa1eb94a40/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.762327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1114/2226] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:30.763922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06978f-c96f-4fab-9bca-d1b23e83c84b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.776399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06978f-c96f-4fab-9bca-d1b23e83c84b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.786019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.787023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.787592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e42e6a-9b5a-460c-bfe3-e3ae34c1645d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafff73d9941412fae6e3c8e92591c9f, user_id=0794c6aadcef4125828554fa1eb94a40, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.787927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.788459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.790696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:30.791021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.792651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e42e6a-9b5a-460c-bfe3-e3ae34c1645d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.792768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.813817 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1115/2227] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:15:30.815329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:30.838241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da669a-7a4b-47ea-a8a6-5eb1f102c4a1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:30.843344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:30.843958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.844406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.844811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.845466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:30.847682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:30.874551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1113/2228] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:30 2022] PUT /identity/v3/projects/dafff73d9941412fae6e3c8e92591c9f/users/0794c6aadcef4125828554fa1eb94a40/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:30.886147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1060-8be1-4d41-b1b4-495d47ad56d8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:30.886635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1060-8be1-4d41-b1b4-495d47ad56d8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:30.886925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1060-8be1-4d41-b1b4-495d47ad56d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:30.956419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e45e1060-8be1-4d41-b1b4-495d47ad56d8 None None] MFA Rules not processed for user `0794c6aadcef4125828554fa1eb94a40`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:30.958006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e45e1060-8be1-4d41-b1b4-495d47ad56d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:31.012168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1116/2229] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:30 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:31.017943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.085431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1114/2230] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:31.088951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:31.092097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1060-8be1-4d41-b1b4-495d47ad56d8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.129328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ce72-f865-40be-a38b-0b101298630e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.136173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.137101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.138130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.138592 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.139451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.142143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.201197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1060-8be1-4d41-b1b4-495d47ad56d8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.206716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.207303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.207527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.208246 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.208976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.210967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.257237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1117/2231] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:31 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:31.261424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.273217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af180ea1-e492-484f-8ef3-eb318f017897 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.280369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.281488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.282880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.283574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.284794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.286876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.307240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1118/2232] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:31.311148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.333791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cafd9-6ab6-481c-ab2d-eba9f77733ce admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.346269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.347067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.347419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.347918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1115/2233] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:31 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:31.349744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.350506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.353003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.354715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.373363 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1119/2234] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:31.380907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.396832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c935dac-017f-4f89-a2e2-e6b3f72e07bb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.453048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:31.466820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87129-ad32-46e9-9262-6ab8c86bea3e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.483179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.483881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.484458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.484844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.486116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.493862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.501035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.501940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.502222 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.502545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] PATH_INFO: `/v3/projects/f0323333989a4014a3be79e8349e1840/users/c835608161474af8a96909d20bc9c94d/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.566264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0323333989a4014a3be79e8349e1840, user_id=c835608161474af8a96909d20bc9c94d, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.572010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.683756 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1116/2235] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:31 2022] POST /identity/v3/users => generated 293 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:31.684759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.688335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:31.706662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787cbb8-ccba-4121-a99a-fca17cba0cfc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.711973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.712618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.712963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.713241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] PATH_INFO: `/v3/projects/1ea5eb8c0fcb4029b8083026bf20d08c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.714452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea5eb8c0fcb4029b8083026bf20d08c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.716515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.742826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1120/2236] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:31 2022] PUT /identity/v3/projects/f0323333989a4014a3be79e8349e1840/users/c835608161474af8a96909d20bc9c94d/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:31.742826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:31.850195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab07ee-ee53-4695-ba11-b55f7235d9f8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:31.858571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c245a27-08a5-41b4-bfe7-b7059ef72d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:31.859553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245a27-08a5-41b4-bfe7-b7059ef72d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.861191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245a27-08a5-41b4-bfe7-b7059ef72d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.861833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245a27-08a5-41b4-bfe7-b7059ef72d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.862690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c245a27-08a5-41b4-bfe7-b7059ef72d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:31.871272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c245a27-08a5-41b4-bfe7-b7059ef72d08 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:31.894399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1121/2237] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:31.896498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a743419-c2d1-4080-ada2-2bce77f06253 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:31.896790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a743419-c2d1-4080-ada2-2bce77f06253 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:31.897085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a743419-c2d1-4080-ada2-2bce77f06253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:31.902713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea5eb8c0fcb4029b8083026bf20d08c', 'request_id': 'req-94803402-0055-4921-9631-1a0694fb17d0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:31.949087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1a743419-c2d1-4080-ada2-2bce77f06253 None None] MFA Rules not processed for user `c835608161474af8a96909d20bc9c94d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:31.958197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1a743419-c2d1-4080-ada2-2bce77f06253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.007415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea5eb8c0fcb4029b8083026bf20d08c', 'request_id': 'req-94803402-0055-4921-9631-1a0694fb17d0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:32.018517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea5eb8c0fcb4029b8083026bf20d08c', 'request_id': 'req-94803402-0055-4921-9631-1a0694fb17d0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:32.040751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] The token cache is being invalidate because project 1ea5eb8c0fcb4029b8083026bf20d08c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:15:32.041043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:32.041892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-94803402-0055-4921-9631-1a0694fb17d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:15:32.045773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1122/2239] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:32.047004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a743419-c2d1-4080-ada2-2bce77f06253 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.047886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1117/2239] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:15:31 2022] DELETE /identity/v3/projects/1ea5eb8c0fcb4029b8083026bf20d08c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:32.049361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.049748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.050031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.078853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:32.080533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.140723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a743419-c2d1-4080-ada2-2bce77f06253 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.147380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.150060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.150355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.150737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] PATH_INFO: `/v3/projects/6e8e7a983d0f426c91a86c29fba02b01/users/3528c19d12384660818211a9fd16468f/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.157314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1118/2240] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:32.158284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.159730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.187590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8e7a983d0f426c91a86c29fba02b01, user_id=3528c19d12384660818211a9fd16468f, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.191135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.231419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92e8a-91df-48a0-ae6b-7de9782290ea None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.237062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a47d5f-b7ab-4516-9263-0d91200ea69f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.237777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a47d5f-b7ab-4516-9263-0d91200ea69f service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.238059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a47d5f-b7ab-4516-9263-0d91200ea69f service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.238770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a47d5f-b7ab-4516-9263-0d91200ea69f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.240399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14a47d5f-b7ab-4516-9263-0d91200ea69f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.255020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a47d5f-b7ab-4516-9263-0d91200ea69f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.256606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a47d5f-b7ab-4516-9263-0d91200ea69f service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.268487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1119/2241] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:32.272070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c0f8e-2ba0-45d8-b608-3c5c5320ad01 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.272507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c0f8e-2ba0-45d8-b608-3c5c5320ad01 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.272916 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c0f8e-2ba0-45d8-b608-3c5c5320ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.306939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b26c0f8e-2ba0-45d8-b608-3c5c5320ad01 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:32.307062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1123/2242] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:32 2022] PUT /identity/v3/projects/6e8e7a983d0f426c91a86c29fba02b01/users/3528c19d12384660818211a9fd16468f/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:32.307145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b26c0f8e-2ba0-45d8-b608-3c5c5320ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.308231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.310719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.415735 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1120/2243] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:32.424546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c0f8e-2ba0-45d8-b608-3c5c5320ad01 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.444432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c0f8e-2ba0-45d8-b608-3c5c5320ad01 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.448576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088646a3-dde5-4c85-9f7d-ce8651bd6eb9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.457039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.458869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.458869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.458869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.458869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.460615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.462320 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.462962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.463244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.463521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.464251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:15:32.464557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.467220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.472376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1124/2244] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:32.478217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.482080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1121/2245] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:15:32.488619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.490838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f5687-d18d-401e-ba4c-3bbe081fafbf admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.496678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621af4f-2399-4b4d-bed1-2d82456e20f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.502757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621af4f-2399-4b4d-bed1-2d82456e20f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.502757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621af4f-2399-4b4d-bed1-2d82456e20f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.502757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621af4f-2399-4b4d-bed1-2d82456e20f3 admin admin] PATH_INFO: `/v3/projects/6e8e7a983d0f426c91a86c29fba02b01/users/3528c19d12384660818211a9fd16468f/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.502757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621af4f-2399-4b4d-bed1-2d82456e20f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8e7a983d0f426c91a86c29fba02b01, user_id=3528c19d12384660818211a9fd16468f, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.503888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5234a-8dc5-4a33-ba1a-a9fd051e5ea4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.513045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621af4f-2399-4b4d-bed1-2d82456e20f3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.513256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.513256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.513256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.513256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.513558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.517287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.521655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1125/2246] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:32 2022] PUT /identity/v3/projects/6e8e7a983d0f426c91a86c29fba02b01/users/3528c19d12384660818211a9fd16468f/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:32.531345 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.531759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.532170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.553443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:32.576495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] MFA Rules not processed for user `3528c19d12384660818211a9fd16468f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:32.579129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.600650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:32.640370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1122/2247] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:32 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:32.642153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.671888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1126/2248] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:32.740588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.742330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:32.743130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e15473-c9a4-4533-a3ae-9fda685cfc09 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.760110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.760512 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.760803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.761322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.764047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.767611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.837926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1a84a-b5f2-45f4-aa5a-1c78a7251cd4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.897884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1123/2249] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:32 2022] POST /identity/v3/users => generated 290 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:32.900027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:32.913720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00238d-d62a-45fc-9429-afcf3f2ab971 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:32.919670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.920129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:32.924435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.924435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.924735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:32.924818 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.924908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:32.924990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:32.925062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:32.925133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:32.936494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1124/2250] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:32.964475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:33.002379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d7424-63ff-403a-a7e6-391dff99dfbc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:33.015490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3cb1a-6105-4fbe-8a57-456d46b4173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:33.016506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3cb1a-6105-4fbe-8a57-456d46b4173d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:33.017043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3cb1a-6105-4fbe-8a57-456d46b4173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:33.018773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3cb1a-6105-4fbe-8a57-456d46b4173d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:33.019812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3cb1a-6105-4fbe-8a57-456d46b4173d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:33.021546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3cb1a-6105-4fbe-8a57-456d46b4173d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:33.031184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:33.034223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:33.046285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1127/2251] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:15:33.050907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1125/2252] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:33.055849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:33.083394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbd903-e8b6-4530-b513-e4fd13580c0f service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:33.091475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:33.091475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:33.092783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:33.092783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] PATH_INFO: `/v3/projects/56e82ba24a474557af758826743136d9/users/b694f1163ef54b82b86e06bf825c57e4/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:33.165388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e82ba24a474557af758826743136d9, user_id=b694f1163ef54b82b86e06bf825c57e4, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:33.168253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:33.312500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1128/2253] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:33 2022] PUT /identity/v3/projects/56e82ba24a474557af758826743136d9/users/b694f1163ef54b82b86e06bf825c57e4/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:33.321656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:33.321656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:33.321656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:33.327727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:33.356871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:33.359102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:33.378006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:33.434218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f37624-dae0-40a5-91ac-9458e6edb25e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:33.438854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea4cb4-bcf4-4e67-a463-d21a2f7f3770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:33.440023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea4cb4-bcf4-4e67-a463-d21a2f7f3770 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:33.440494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea4cb4-bcf4-4e67-a463-d21a2f7f3770 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:33.440935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea4cb4-bcf4-4e67-a463-d21a2f7f3770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:33.444351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-73ea4cb4-bcf4-4e67-a463-d21a2f7f3770 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:33.497235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1126/2254] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:33.505371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:33.513850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:33.594655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2529d-f319-4e4e-9022-0039b50d22d8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:33.595744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea4cb4-bcf4-4e67-a463-d21a2f7f3770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:33.598993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea4cb4-bcf4-4e67-a463-d21a2f7f3770 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:33.613331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1129/2255] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:33.613934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4724b-3a1c-403f-b078-707fd0ecd9b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:33.614558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4724b-3a1c-403f-b078-707fd0ecd9b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:33.614834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4724b-3a1c-403f-b078-707fd0ecd9b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:33.615103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4724b-3a1c-403f-b078-707fd0ecd9b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:33.628514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4724b-3a1c-403f-b078-707fd0ecd9b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:33.639675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4724b-3a1c-403f-b078-707fd0ecd9b2 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:33.653078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1127/2256] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:33.851249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:34.208555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:34.209089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:34.211148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:34.238352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] MFA Rules not processed for user `c835608161474af8a96909d20bc9c94d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:34.239785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:34.255882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:34.336346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1130/2257] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:35.469124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d181d9f-435a-42d0-82cd-947e710e3c45 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:35.469700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d181d9f-435a-42d0-82cd-947e710e3c45 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:35.469700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d181d9f-435a-42d0-82cd-947e710e3c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:35.502972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0d181d9f-435a-42d0-82cd-947e710e3c45 None None] MFA Rules not processed for user `7796ec614f2545c29c2739d3dfd889f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:35.504190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0d181d9f-435a-42d0-82cd-947e710e3c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:35.595441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1128/2258] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:35.600040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:35.602482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:35.619890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8719d-d887-42f5-946e-120a76ffc8e8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:35.624950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:35.625937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:35.626220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:35.626495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:35.627275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:35.629213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:35.762850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1131/2259] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:35 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:35.767188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d181d9f-435a-42d0-82cd-947e710e3c45 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:35.793816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d181d9f-435a-42d0-82cd-947e710e3c45 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:35.980614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:35.982250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:35.982588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:35.982946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:35.984185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:35.989269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:36.116751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1129/2260] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:35 2022] POST /identity/v3/users => generated 292 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:36.120081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:36.132807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0bc29-ef07-442b-9555-4683c04c90d9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:36.139921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:36.141691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:36.141691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:36.141924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:36.142721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:36.144997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:36.159739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1132/2261] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:15:36.162915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:36.169817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:36.204667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac1c97-44af-42e0-a622-0e45bbfd85f6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:36.221243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:36.222453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:36.223141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:36.223662 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] PATH_INFO: `/v3/projects/e490f07e81604d68a8aa6ccc2b3ee34d/users/62804425888c4f3d83991e3e3b678d30/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:36.255010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e490f07e81604d68a8aa6ccc2b3ee34d, user_id=62804425888c4f3d83991e3e3b678d30, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:36.257894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:36.343624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1130/2262] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:36 2022] PUT /identity/v3/projects/e490f07e81604d68a8aa6ccc2b3ee34d/users/62804425888c4f3d83991e3e3b678d30/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:36.348616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:36.365939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20daaf-7d3a-4319-bb7c-7977b5acf50c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:36.423040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:36.581801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cc3c3-3cf1-4783-a400-767f4c24b237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:36.583336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cc3c3-3cf1-4783-a400-767f4c24b237 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:36.583717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cc3c3-3cf1-4783-a400-767f4c24b237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:36.584092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cc3c3-3cf1-4783-a400-767f4c24b237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:36.585243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cc3c3-3cf1-4783-a400-767f4c24b237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:36.587159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cc3c3-3cf1-4783-a400-767f4c24b237 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:36.607717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1133/2263] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:36.609745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:36.623366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01592d1-03ca-4671-baff-026f1a65935b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:36.628935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:36.629793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:36.630131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:36.630550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] PATH_INFO: `/v3/projects/e490f07e81604d68a8aa6ccc2b3ee34d/users/62804425888c4f3d83991e3e3b678d30/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:36.632778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e490f07e81604d68a8aa6ccc2b3ee34d, user_id=62804425888c4f3d83991e3e3b678d30, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:36.635526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:36.646639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1131/2264] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:36 2022] PUT /identity/v3/projects/e490f07e81604d68a8aa6ccc2b3ee34d/users/62804425888c4f3d83991e3e3b678d30/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:41.927553 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93140f3-4cee-4394-985a-4608b42e2104 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:41.928455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93140f3-4cee-4394-985a-4608b42e2104 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:41.928973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93140f3-4cee-4394-985a-4608b42e2104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:42.010490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b93140f3-4cee-4394-985a-4608b42e2104 None None] MFA Rules not processed for user `b694f1163ef54b82b86e06bf825c57e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:42.010490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b93140f3-4cee-4394-985a-4608b42e2104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:42.201827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1134/2265] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:42.207063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:42.419274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628838a2-a753-4f1d-8faf-6c262c233d07 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:42.425674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:42.426306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:42.426588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:42.426914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:42.429004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:42.432577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:42.553716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1132/2266] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:15:42 2022] POST /identity/v3/projects => generated 373 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:15:42.557291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93140f3-4cee-4394-985a-4608b42e2104 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:42.573283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93140f3-4cee-4394-985a-4608b42e2104 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:42.765159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:42.766303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:42.766833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:42.768397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:42.769276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:42.775812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:42.897230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1135/2267] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:15:42 2022] POST /identity/v3/users => generated 293 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:15:42.901204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:42.928851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759925b-4eec-48be-916b-56d0eb8e1d3b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:42.949436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:42.954169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:42.954169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:42.954169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:42.954169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:42.954169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:42.994307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:42.994908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1133/2268] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:43.023596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133fef0-d7e1-4cf0-81c3-99fefe434c31 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:43.040470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:43.041313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:43.041659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:43.042048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] PATH_INFO: `/v3/projects/89ffdbbdedc24fbca2fbb10c2d2fd466/users/c75af86aac424ddda3c8afd3ba30b4d5/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:43.088778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ffdbbdedc24fbca2fbb10c2d2fd466, user_id=c75af86aac424ddda3c8afd3ba30b4d5, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:43.091753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:43.190450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1136/2269] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:42 2022] PUT /identity/v3/projects/89ffdbbdedc24fbca2fbb10c2d2fd466/users/c75af86aac424ddda3c8afd3ba30b4d5/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:15:43.195910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:43.212323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0eb47-b0a2-4eb0-99cc-06681c1da2a3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:43.456813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3524a86-b81b-4b33-808d-7de126df880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:43.458885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3524a86-b81b-4b33-808d-7de126df880c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:43.459368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3524a86-b81b-4b33-808d-7de126df880c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:43.459807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3524a86-b81b-4b33-808d-7de126df880c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:43.460639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3524a86-b81b-4b33-808d-7de126df880c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:43.468551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3524a86-b81b-4b33-808d-7de126df880c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:43.515352 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1134/2270] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:15:43.520198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:43.548603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5d1a2-4a75-492a-9d87-4c0f62479dda admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:43.559653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:43.560809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:43.561311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:43.561851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] PATH_INFO: `/v3/projects/89ffdbbdedc24fbca2fbb10c2d2fd466/users/c75af86aac424ddda3c8afd3ba30b4d5/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:43.566495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ffdbbdedc24fbca2fbb10c2d2fd466, user_id=c75af86aac424ddda3c8afd3ba30b4d5, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:43.571983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:43.588182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:43.619956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1137/2271] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:15:43 2022] PUT /identity/v3/projects/89ffdbbdedc24fbca2fbb10c2d2fd466/users/c75af86aac424ddda3c8afd3ba30b4d5/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:15:46.234767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0116c4-50c8-4415-a975-ada29039c5bc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:46.235515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0116c4-50c8-4415-a975-ada29039c5bc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:46.235515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0116c4-50c8-4415-a975-ada29039c5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:46.344596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cb0116c4-50c8-4415-a975-ada29039c5bc None None] MFA Rules not processed for user `62804425888c4f3d83991e3e3b678d30`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:46.348329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cb0116c4-50c8-4415-a975-ada29039c5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:46.422828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:15:46.602078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:46.603848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:46.618048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1135/2272] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:46.801519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223326d-c02b-42a8-b938-349e230683fd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:46.815372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463ddc7-8505-45c6-bade-f2cab19ce43f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:46.819985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463ddc7-8505-45c6-bade-f2cab19ce43f service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:46.820370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463ddc7-8505-45c6-bade-f2cab19ce43f service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:46.820671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463ddc7-8505-45c6-bade-f2cab19ce43f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:46.904737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463ddc7-8505-45c6-bade-f2cab19ce43f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:46.907634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463ddc7-8505-45c6-bade-f2cab19ce43f service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:46.919254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1138/2273] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:15:52.873877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d86e5-f69a-4320-b827-59a414145269 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:52.874786 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d86e5-f69a-4320-b827-59a414145269 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:52.875839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d86e5-f69a-4320-b827-59a414145269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:52.897111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0a4d86e5-f69a-4320-b827-59a414145269 None None] MFA Rules not processed for user `7796ec614f2545c29c2739d3dfd889f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:52.900264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0a4d86e5-f69a-4320-b827-59a414145269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:52.912267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1136/2274] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:15:54.939356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986a2fb-84c6-432f-9d39-fcb311f0fecb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:54.942086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986a2fb-84c6-432f-9d39-fcb311f0fecb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:54.942577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986a2fb-84c6-432f-9d39-fcb311f0fecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:55.009497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2986a2fb-84c6-432f-9d39-fcb311f0fecb None None] MFA Rules not processed for user `c75af86aac424ddda3c8afd3ba30b4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:15:55.010956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2986a2fb-84c6-432f-9d39-fcb311f0fecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:15:55.107075 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1139/2275] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:15:55.122109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d86e5-f69a-4320-b827-59a414145269 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:15:55.137672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d86e5-f69a-4320-b827-59a414145269 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:15:55.145418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864df3a1-3d51-4ac9-a4a8-e3716bf2c936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:15:55.146482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864df3a1-3d51-4ac9-a4a8-e3716bf2c936 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:15:55.147093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864df3a1-3d51-4ac9-a4a8-e3716bf2c936 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:15:55.147626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864df3a1-3d51-4ac9-a4a8-e3716bf2c936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:15:55.220130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864df3a1-3d51-4ac9-a4a8-e3716bf2c936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:15:55.221761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864df3a1-3d51-4ac9-a4a8-e3716bf2c936 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:15:55.230754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1137/2276] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:15:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:05.103880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:05.226627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=49 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:21.459837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:21.460715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:21.461003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:21.481876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] MFA Rules not processed for user `6e5c22b631df4600b9f920120b67fc67`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:21.484120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:21.569879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1140/2277] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:21.927977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74093e45-92fb-4fac-9496-712e636be6da None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:21.932770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74093e45-92fb-4fac-9496-712e636be6da None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:21.932770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74093e45-92fb-4fac-9496-712e636be6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:21.952194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-74093e45-92fb-4fac-9496-712e636be6da None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:21.953622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-74093e45-92fb-4fac-9496-712e636be6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:22.035360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1138/2278] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:22.092058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:22.093418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:22.172995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153031b9-bb80-4882-9f06-ea86f19571c8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:22.179076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:22.179706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:22.179995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:22.180272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:22.189071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:22.190910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:22.199421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1141/2279] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:16:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:16:23.596616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94615f-3923-4e06-a6ae-c9fdf0095a69 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:23.597243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94615f-3923-4e06-a6ae-c9fdf0095a69 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:23.597243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94615f-3923-4e06-a6ae-c9fdf0095a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:23.635296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3a94615f-3923-4e06-a6ae-c9fdf0095a69 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:23.637087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3a94615f-3923-4e06-a6ae-c9fdf0095a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:23.648103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1139/2280] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:23.676755 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:23.690926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5141-b469-49e7-ac1b-5191f167f1f8 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:23.701030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:23.701884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:23.702534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:23.703059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:23.748485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:23.759001 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:23.778441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1142/2281] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:16:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:16:25.773274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f798b-4385-4bc1-8df5-6e336ea8578f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:25.774347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f798b-4385-4bc1-8df5-6e336ea8578f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:25.775123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f798b-4385-4bc1-8df5-6e336ea8578f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:25.823350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d70f798b-4385-4bc1-8df5-6e336ea8578f None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:25.824604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d70f798b-4385-4bc1-8df5-6e336ea8578f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:25.864300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1140/2282] 10.222.0.39 () {58 vars in 967 bytes} [Sun Sep 4 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:25.877871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:25.959964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9732c1-6b00-484a-98ca-6fce006aadea service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:25.980930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e50c2-6e7b-41b9-9bd0-b156a67ae827 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:25.982369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e50c2-6e7b-41b9-9bd0-b156a67ae827 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:25.982853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e50c2-6e7b-41b9-9bd0-b156a67ae827 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:25.983294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e50c2-6e7b-41b9-9bd0-b156a67ae827 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:26.005885 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e50c2-6e7b-41b9-9bd0-b156a67ae827 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:26.011030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e50c2-6e7b-41b9-9bd0-b156a67ae827 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:26.046396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1143/2283] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:16:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:16:27.365886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f2af1-230d-4fb7-ad54-ec48fd6d2321 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:27.366491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f2af1-230d-4fb7-ad54-ec48fd6d2321 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:27.366491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f2af1-230d-4fb7-ad54-ec48fd6d2321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:27.398176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ca0f2af1-230d-4fb7-ad54-ec48fd6d2321 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:27.406732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca0f2af1-230d-4fb7-ad54-ec48fd6d2321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:27.418208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1141/2284] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:16:27.443170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46481f-7eba-45d9-bbf5-5953dae43d37 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:27.446277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46481f-7eba-45d9-bbf5-5953dae43d37 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:27.446756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46481f-7eba-45d9-bbf5-5953dae43d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:27.450075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f2af1-230d-4fb7-ad54-ec48fd6d2321 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:27.470161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f2af1-230d-4fb7-ad54-ec48fd6d2321 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:27.499211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b36a98-999f-4b86-8189-55cf56ac1fa3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:27.501960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b36a98-999f-4b86-8189-55cf56ac1fa3 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:27.501960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b36a98-999f-4b86-8189-55cf56ac1fa3 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:27.501960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b36a98-999f-4b86-8189-55cf56ac1fa3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:27.509427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bf46481f-7eba-45d9-bbf5-5953dae43d37 None None] MFA Rules not processed for user `a9d55c52717546b09d93d956a29f49a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:27.517304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bf46481f-7eba-45d9-bbf5-5953dae43d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:27.526685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b36a98-999f-4b86-8189-55cf56ac1fa3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:27.529277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b36a98-999f-4b86-8189-55cf56ac1fa3 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:27.567249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1142/2285] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:16:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 04 16:16:27.600918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1144/2286] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:28.622992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e415b-f5e6-4a61-bb06-3edafac9a8e4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:28.625391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e415b-f5e6-4a61-bb06-3edafac9a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:28.625391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e415b-f5e6-4a61-bb06-3edafac9a8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:28.654460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5c2e415b-f5e6-4a61-bb06-3edafac9a8e4 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:28.654460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5c2e415b-f5e6-4a61-bb06-3edafac9a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:28.663374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1143/2287] 10.222.0.39 () {58 vars in 967 bytes} [Sun Sep 4 16:16:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:28.729307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46481f-7eba-45d9-bbf5-5953dae43d37 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:28.752928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46481f-7eba-45d9-bbf5-5953dae43d37 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:28.759873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:28.760844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:28.761297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:28.761787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:28.771935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:28.773765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:28.778957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:28.783777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1145/2288] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:16:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:16:30.941853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e584033-4ff8-41a8-87fc-ebdeec054bee None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:30.942782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e584033-4ff8-41a8-87fc-ebdeec054bee None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:30.943219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e584033-4ff8-41a8-87fc-ebdeec054bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:30.974746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2e584033-4ff8-41a8-87fc-ebdeec054bee None None] MFA Rules not processed for user `b694f1163ef54b82b86e06bf825c57e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:30.976598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2e584033-4ff8-41a8-87fc-ebdeec054bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:31.003607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1144/2289] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:34.131420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:34.139639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:34.373415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab43b2c-7d61-425c-ac77-0f255b618cd1 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:34.397450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:34.399265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:34.399765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:34.400257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] PATH_INFO: `/v3/users/c835608161474af8a96909d20bc9c94d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:34.401871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c835608161474af8a96909d20bc9c94d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:34.405615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:34.774924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c835608161474af8a96909d20bc9c94d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:34.776158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c835608161474af8a96909d20bc9c94d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:34.802915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c835608161474af8a96909d20bc9c94d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:34.896033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c835608161474af8a96909d20bc9c94d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:34.901731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c835608161474af8a96909d20bc9c94d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:34.933097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1146/2290] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:16:34 2022] DELETE /identity/v3/users/c835608161474af8a96909d20bc9c94d => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:34.938067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:34.938605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:34.938890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:34.982071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:34.983625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:35.096364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1145/2291] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:35.109174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:35.110973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:35.237168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25dcc0-a5a7-49f4-b8ce-8f2f4ca9d7d0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:35.250902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:35.251541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:35.252596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:35.253213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:35.280203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:35.281809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:35.319620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1147/2292] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:35.632092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:35.634386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:35.685356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2a913-58f8-4103-bf3b-d0a44bff561d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:35.696369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:35.697434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:35.711717 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:35.713814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] PATH_INFO: `/v3/projects/f0323333989a4014a3be79e8349e1840` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:35.716584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0323333989a4014a3be79e8349e1840)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:35.725025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:35.936008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0323333989a4014a3be79e8349e1840', 'request_id': 'req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:35.957504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:36.068706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0323333989a4014a3be79e8349e1840', 'request_id': 'req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:36.076285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0323333989a4014a3be79e8349e1840', 'request_id': 'req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:36.118878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] The token cache is being invalidate because project f0323333989a4014a3be79e8349e1840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:16:36.119253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:36.120021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f83ff4b4-4934-470e-ac2a-22ed19eb80ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:36.134577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1146/2293] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:16:35 2022] DELETE /identity/v3/projects/f0323333989a4014a3be79e8349e1840 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:39.946773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:39.948296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:39.962942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4d7a7-306d-4bcf-b4f9-004e370a34bc service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:39.999053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:40.043692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:40.044331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:40.044607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:40.044876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] PATH_INFO: `/v3/users/0794c6aadcef4125828554fa1eb94a40` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:40.046063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0794c6aadcef4125828554fa1eb94a40)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:40.047998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:40.277064 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0794c6aadcef4125828554fa1eb94a40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:40.277746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0794c6aadcef4125828554fa1eb94a40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:40.290264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0794c6aadcef4125828554fa1eb94a40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:40.358320 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0794c6aadcef4125828554fa1eb94a40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:40.359428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0794c6aadcef4125828554fa1eb94a40'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:40.372516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1148/2294] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:16:39 2022] DELETE /identity/v3/users/0794c6aadcef4125828554fa1eb94a40 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:40.377828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:40.378412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:40.378586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:40.398427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:40.401866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:40.460652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1147/2295] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:40.472250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:40.474055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:40.548202 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797846-db63-4ebf-930d-1751c21508c7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:40.552651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:40.553269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:40.553554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:40.553865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:40.563451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:40.565369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:40.589772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1149/2296] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:41.037999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:41.037999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:41.052050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb39776-49bb-445e-a1af-6b913763de09 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:41.066738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:41.078690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:41.079143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:41.079501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] PATH_INFO: `/v3/projects/dafff73d9941412fae6e3c8e92591c9f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:41.084009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dafff73d9941412fae6e3c8e92591c9f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:41.086255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:41.203499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dafff73d9941412fae6e3c8e92591c9f', 'request_id': 'req-2af64e0d-7055-4dd4-97dd-091148fe2316'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.270368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dafff73d9941412fae6e3c8e92591c9f', 'request_id': 'req-2af64e0d-7055-4dd4-97dd-091148fe2316'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.287153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dafff73d9941412fae6e3c8e92591c9f', 'request_id': 'req-2af64e0d-7055-4dd4-97dd-091148fe2316'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.310092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] The token cache is being invalidate because project dafff73d9941412fae6e3c8e92591c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:16:41.310092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.316115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.320681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1148/2297] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:16:41 2022] DELETE /identity/v3/projects/dafff73d9941412fae6e3c8e92591c9f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:41.324604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:41.348534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f3b3d-372b-4f4e-9e7d-ba1c4d9ced7f service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:41.401703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:41.424523 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:41.425213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:41.426008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:41.426008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] PATH_INFO: `/v3/users/3528c19d12384660818211a9fd16468f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:41.427051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3528c19d12384660818211a9fd16468f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:41.429086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:41.695511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3528c19d12384660818211a9fd16468f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.696528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3528c19d12384660818211a9fd16468f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.709895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3528c19d12384660818211a9fd16468f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.790763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3528c19d12384660818211a9fd16468f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.790763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a837afc7-981b-4f5f-adee-eee3d8c32181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3528c19d12384660818211a9fd16468f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:41.819248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1150/2298] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:16:41 2022] DELETE /identity/v3/users/3528c19d12384660818211a9fd16468f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:42.616073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:42.618640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:42.640030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af64e0d-7055-4dd4-97dd-091148fe2316 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:42.710749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:42.711543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:42.711908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:42.712370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] PATH_INFO: `/v3/projects/6e8e7a983d0f426c91a86c29fba02b01` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:42.713746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8e7a983d0f426c91a86c29fba02b01)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:42.715889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:42.726726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:42.866095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8e7a983d0f426c91a86c29fba02b01', 'request_id': 'req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:42.976334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8e7a983d0f426c91a86c29fba02b01', 'request_id': 'req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:42.984247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8e7a983d0f426c91a86c29fba02b01', 'request_id': 'req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:43.000301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] The token cache is being invalidate because project 6e8e7a983d0f426c91a86c29fba02b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:16:43.000893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:43.003228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-250cef8a-0cf8-4cc6-be3f-29bf1ed342ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:43.006602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1149/2299] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:16:42 2022] DELETE /identity/v3/projects/6e8e7a983d0f426c91a86c29fba02b01 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:43.011913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b687b7-13de-4a6f-aca1-8c053af2e497 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.012763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b687b7-13de-4a6f-aca1-8c053af2e497 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.013204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b687b7-13de-4a6f-aca1-8c053af2e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.037043 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-03b687b7-13de-4a6f-aca1-8c053af2e497 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:43.038673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03b687b7-13de-4a6f-aca1-8c053af2e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:43.104267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1151/2300] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:43.112814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52add3f-c56b-450e-a447-36ff71d21832 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.113109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52add3f-c56b-450e-a447-36ff71d21832 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.113837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52add3f-c56b-450e-a447-36ff71d21832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.136440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b52add3f-c56b-450e-a447-36ff71d21832 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:43.137942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b52add3f-c56b-450e-a447-36ff71d21832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:43.169449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1150/2301] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:43.175078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b687b7-13de-4a6f-aca1-8c053af2e497 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:43.187398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b687b7-13de-4a6f-aca1-8c053af2e497 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:43.195388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:43.196114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.196507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.196845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.197585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:16:43.197963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:43.199637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:43.212434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1152/2302] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:16:43.216946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52add3f-c56b-450e-a447-36ff71d21832 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:43.227327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52add3f-c56b-450e-a447-36ff71d21832 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:43.231994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:43.232609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.232882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.233190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.233917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:43.235681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:43.352549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1151/2303] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:16:43 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:16:43.356727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:43.369441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e567316-9cd7-4eb1-b741-b7e05100dca3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:43.431430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:43.441910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:43.442583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.443035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.443310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.443969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:43.445945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:43.567686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1153/2304] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:16:43 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:16:43.573748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:43.588878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48150b07-322f-4324-8b2f-c8f26d1edf8f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:43.594813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:43.594813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.594813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.595111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.597507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:43.598743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:43.614732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1152/2305] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:16:43.618876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:43.630187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37851a-78a0-4fc0-a9c4-1edf7fc71e92 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:43.636140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3641d7d-0325-4efa-8c79-f8c7a98a961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:43.637180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3641d7d-0325-4efa-8c79-f8c7a98a961a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.637742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3641d7d-0325-4efa-8c79-f8c7a98a961a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.638236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3641d7d-0325-4efa-8c79-f8c7a98a961a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.639766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3641d7d-0325-4efa-8c79-f8c7a98a961a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:43.645402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3641d7d-0325-4efa-8c79-f8c7a98a961a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:43.657603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1154/2306] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:16:43.662570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:43.678684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac0eab-15cc-4fe4-a08d-d268ef9cda04 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:43.683727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:43.684391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.684671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.684945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] PATH_INFO: `/v3/projects/966c7920aaa645aea3c92402322a917c/users/4b16e216ae3f4d39a0f0f890d9f0dff7/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.701591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966c7920aaa645aea3c92402322a917c, user_id=4b16e216ae3f4d39a0f0f890d9f0dff7, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:43.704249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:43.781608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1153/2307] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:16:43 2022] PUT /identity/v3/projects/966c7920aaa645aea3c92402322a917c/users/4b16e216ae3f4d39a0f0f890d9f0dff7/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:43.794634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5efc0-2585-4ffb-824a-85333b1f2dfa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.794902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5efc0-2585-4ffb-824a-85333b1f2dfa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.795190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5efc0-2585-4ffb-824a-85333b1f2dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.824045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-def5efc0-2585-4ffb-824a-85333b1f2dfa None None] MFA Rules not processed for user `4b16e216ae3f4d39a0f0f890d9f0dff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:43.825297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-def5efc0-2585-4ffb-824a-85333b1f2dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:43.894008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1155/2308] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:16:43.898010 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:43.969881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed65e8-0742-4026-a3fb-a8344e413845 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:43.974932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:43.975541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:43.975824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:43.976101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:43.976737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:43.978686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:44.074226 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1154/2309] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:16:43 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:16:44.078254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5efc0-2585-4ffb-824a-85333b1f2dfa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:44.090500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5efc0-2585-4ffb-824a-85333b1f2dfa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:44.148351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:44.148962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.149242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.149515 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:44.150189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:44.152335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:44.215486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1156/2310] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:16:44 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:16:44.219666 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:44.232209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388c493-9013-4972-9fa2-043d051e9e93 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:44.241381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:44.242045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.244141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.244141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:44.244141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:44.244934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:44.258659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1155/2311] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:16:44.263054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:44.276329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88095b70-beb6-40f2-8d37-12d9a7053a01 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:44.284361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:44.284361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.284361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.284361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] PATH_INFO: `/v3/projects/93978cc4d1794e5d9c9419cbe70d680e/users/2ab71ef21b2040059fe683c26203cdf0/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:44.304204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93978cc4d1794e5d9c9419cbe70d680e, user_id=2ab71ef21b2040059fe683c26203cdf0, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:44.311234 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:44.394467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1157/2312] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:16:44 2022] PUT /identity/v3/projects/93978cc4d1794e5d9c9419cbe70d680e/users/2ab71ef21b2040059fe683c26203cdf0/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:44.398458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:44.409992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6bc87-4242-4626-90b9-563ac9140383 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:44.414022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:44.512575 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4479f1a-d4b3-4abb-b355-f517a4f5f5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:44.513268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479f1a-d4b3-4abb-b355-f517a4f5f5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.513895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479f1a-d4b3-4abb-b355-f517a4f5f5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.514263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479f1a-d4b3-4abb-b355-f517a4f5f5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:44.515069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4479f1a-d4b3-4abb-b355-f517a4f5f5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:44.517972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4479f1a-d4b3-4abb-b355-f517a4f5f5b3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:44.529599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1156/2313] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:16:44.532546 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:44.547251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fe167-304e-4e68-8bd5-1be37d24cea5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:44.563418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:44.564178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.565962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.565962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] PATH_INFO: `/v3/projects/93978cc4d1794e5d9c9419cbe70d680e/users/2ab71ef21b2040059fe683c26203cdf0/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:44.567812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93978cc4d1794e5d9c9419cbe70d680e, user_id=2ab71ef21b2040059fe683c26203cdf0, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:44.570947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:44.586959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1158/2314] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:16:44 2022] PUT /identity/v3/projects/93978cc4d1794e5d9c9419cbe70d680e/users/2ab71ef21b2040059fe683c26203cdf0/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:44.597089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.597371 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.597686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:44.639936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] MFA Rules not processed for user `2ab71ef21b2040059fe683c26203cdf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:44.641899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:44.723370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1157/2315] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:44.734684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:44.736634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:44.751186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:44.815731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8134bd0-7a66-4562-b537-f7a1e5675104 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:44.820714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:44.821408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.821829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.822261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:44.832233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:44.833797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:44.844019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1159/2316] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:44.945665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:44.947216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:44.959491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2e11c-6b3a-44d5-980b-00ca7455d392 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:44.963390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc357487-0440-4f9d-8a62-afe020f082bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:44.964022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc357487-0440-4f9d-8a62-afe020f082bc service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:44.964303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc357487-0440-4f9d-8a62-afe020f082bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:44.964595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc357487-0440-4f9d-8a62-afe020f082bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:45.027098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc357487-0440-4f9d-8a62-afe020f082bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:45.028540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc357487-0440-4f9d-8a62-afe020f082bc service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:45.039278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1158/2317] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:46.458160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:46.461083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:46.585675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3eb358-ae8e-4295-88df-6d9c2184fd96 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:46.592092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5c447-af2a-44a9-841d-f1886bc06a60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:46.593807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5c447-af2a-44a9-841d-f1886bc06a60 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:46.593807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5c447-af2a-44a9-841d-f1886bc06a60 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:46.593807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5c447-af2a-44a9-841d-f1886bc06a60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:46.596076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fed5c447-af2a-44a9-841d-f1886bc06a60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:46.625404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5c447-af2a-44a9-841d-f1886bc06a60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:46.627634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5c447-af2a-44a9-841d-f1886bc06a60 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:46.655226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1160/2318] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:46.659154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721c310-2eb7-4e22-8bca-fdd00514f841 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:46.660057 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721c310-2eb7-4e22-8bca-fdd00514f841 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:46.660508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721c310-2eb7-4e22-8bca-fdd00514f841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:46.684990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0721c310-2eb7-4e22-8bca-fdd00514f841 None None] MFA Rules not processed for user `a8925da59209424fbd6375f056aee752`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:46.686785 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0721c310-2eb7-4e22-8bca-fdd00514f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:46.715716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:46.793444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1159/2319] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:16:46.843230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed82d0-a775-42cc-b375-9ba3e23d3a2d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:46.843861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed82d0-a775-42cc-b375-9ba3e23d3a2d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:46.844230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed82d0-a775-42cc-b375-9ba3e23d3a2d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:46.845959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1161/2320] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:16:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:16:46.852173 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c6cdd-5f16-46cc-97f8-321bf58ce48c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:46.852749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c6cdd-5f16-46cc-97f8-321bf58ce48c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:46.853078 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c6cdd-5f16-46cc-97f8-321bf58ce48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:46.878205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1d8c6cdd-5f16-46cc-97f8-321bf58ce48c None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:46.880203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1d8c6cdd-5f16-46cc-97f8-321bf58ce48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:46.987487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1160/2321] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:16:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:46.999723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed82d0-a775-42cc-b375-9ba3e23d3a2d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:47.001707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c5ed82d0-a775-42cc-b375-9ba3e23d3a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:47.066680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed82d0-a775-42cc-b375-9ba3e23d3a2d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:47.070628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eba7a-d2ad-4044-af66-f076dcf85ace service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95204bd338a4a99a510a67c4ab97532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:47.073689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eba7a-d2ad-4044-af66-f076dcf85ace service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:47.073689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eba7a-d2ad-4044-af66-f076dcf85ace service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:47.073689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eba7a-d2ad-4044-af66-f076dcf85ace service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:47.084811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eba7a-d2ad-4044-af66-f076dcf85ace service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:47.086505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eba7a-d2ad-4044-af66-f076dcf85ace service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:47.120192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1162/2322] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:47.325302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5040dee-a99e-4cfe-8655-183a52619de2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:47.325895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5040dee-a99e-4cfe-8655-183a52619de2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:47.326325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5040dee-a99e-4cfe-8655-183a52619de2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:47.328456 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1161/2323] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:16:47.336375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7358d0-321d-440c-a457-a81a9c9f412e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:47.337229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7358d0-321d-440c-a457-a81a9c9f412e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:47.337855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7358d0-321d-440c-a457-a81a9c9f412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:47.360721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6d7358d0-321d-440c-a457-a81a9c9f412e None None] MFA Rules not processed for user `93ef027007184b8180325d6fb26324a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:47.362055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6d7358d0-321d-440c-a457-a81a9c9f412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:47.413038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:47.425874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1163/2324] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:16:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:47.438451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5040dee-a99e-4cfe-8655-183a52619de2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:47.440436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e5040dee-a99e-4cfe-8655-183a52619de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:47.455799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5040dee-a99e-4cfe-8655-183a52619de2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:47.460008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73e45b-b55f-4dc7-b4c4-9ef8b6f168a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:47.460644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e45b-b55f-4dc7-b4c4-9ef8b6f168a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:47.460929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e45b-b55f-4dc7-b4c4-9ef8b6f168a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:47.461206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e45b-b55f-4dc7-b4c4-9ef8b6f168a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:47.469658 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73e45b-b55f-4dc7-b4c4-9ef8b6f168a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:47.471517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73e45b-b55f-4dc7-b4c4-9ef8b6f168a0 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:47.482176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1162/2325] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:47.576670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559721-4dbd-4297-b36b-6d66c66944f7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:47.577238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559721-4dbd-4297-b36b-6d66c66944f7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:47.577536 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559721-4dbd-4297-b36b-6d66c66944f7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:47.579056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1164/2326] 10.222.0.39 () {54 vars in 921 bytes} [Sun Sep 4 16:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 16:16:47.593189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc1c58-7995-4501-9727-1577e95cfaf3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:47.593189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc1c58-7995-4501-9727-1577e95cfaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:47.593189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc1c58-7995-4501-9727-1577e95cfaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:47.609200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-25bc1c58-7995-4501-9727-1577e95cfaf3 None None] MFA Rules not processed for user `46fd540df66143b2b768cc8d9f4beed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:47.610548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-25bc1c58-7995-4501-9727-1577e95cfaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:47.622276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1163/2327] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Sep 4 16:16:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:47.634870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559721-4dbd-4297-b36b-6d66c66944f7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:47.648723 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559721-4dbd-4297-b36b-6d66c66944f7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:47.652943 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a49aa-7d8e-45ed-bcc6-33e4f5842584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:47.653549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a49aa-7d8e-45ed-bcc6-33e4f5842584 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:47.653962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a49aa-7d8e-45ed-bcc6-33e4f5842584 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:47.654205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a49aa-7d8e-45ed-bcc6-33e4f5842584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:47.668366 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a49aa-7d8e-45ed-bcc6-33e4f5842584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:47.682630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a49aa-7d8e-45ed-bcc6-33e4f5842584 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:47.699659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1165/2328] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:50.055718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc1c58-7995-4501-9727-1577e95cfaf3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:50.148468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc1c58-7995-4501-9727-1577e95cfaf3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:50.154714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:50.155708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:50.156143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:50.156620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] PATH_INFO: `/v3/users/4b16e216ae3f4d39a0f0f890d9f0dff7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:50.159150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b16e216ae3f4d39a0f0f890d9f0dff7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:50.161536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:50.378002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:50.380038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b16e216ae3f4d39a0f0f890d9f0dff7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:50.380730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b16e216ae3f4d39a0f0f890d9f0dff7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:50.461259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b16e216ae3f4d39a0f0f890d9f0dff7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:50.463451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b16e216ae3f4d39a0f0f890d9f0dff7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:50.492296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b16e216ae3f4d39a0f0f890d9f0dff7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:50.510080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1164/2329] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:16:50 2022] DELETE /identity/v3/users/4b16e216ae3f4d39a0f0f890d9f0dff7 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:50.514611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:50.515121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:50.515358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:50.534835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:50.535961 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:50.595058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1166/2330] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:50.604956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:50.606407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:50.666718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37442dce-ff29-47b4-ab93-730fb23131a6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:50.670844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:50.671451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:50.671734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:50.672009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:50.682363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:50.684123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:50.694287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1165/2331] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:51.079811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:51.081464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:51.103274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de21183-89f8-4581-940c-fd0ad5cfcefc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:51.113914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:51.114593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:51.114882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:51.115162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] PATH_INFO: `/v3/projects/966c7920aaa645aea3c92402322a917c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:51.116405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] RBAC: Authorizing `identity:delete_project(project_id=966c7920aaa645aea3c92402322a917c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:51.118495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:51.289506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966c7920aaa645aea3c92402322a917c', 'request_id': 'req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.296305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966c7920aaa645aea3c92402322a917c', 'request_id': 'req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.304382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966c7920aaa645aea3c92402322a917c', 'request_id': 'req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.397664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] The token cache is being invalidate because project 966c7920aaa645aea3c92402322a917c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:16:51.397970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.398618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.400122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1167/2332] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:16:51 2022] DELETE /identity/v3/projects/966c7920aaa645aea3c92402322a917c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:16:51.404053 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:51.414941 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b9c87-f910-43ca-b22c-7884a8411195 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:51.482850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:51.483495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:51.483780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:51.484054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] PATH_INFO: `/v3/users/2ab71ef21b2040059fe683c26203cdf0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:51.485306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab71ef21b2040059fe683c26203cdf0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:51.488361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:51.712334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab71ef21b2040059fe683c26203cdf0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.713205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab71ef21b2040059fe683c26203cdf0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.803705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab71ef21b2040059fe683c26203cdf0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.804331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab71ef21b2040059fe683c26203cdf0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.815716 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:51.817938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6bc2bf85-1943-433f-b227-09dc91673414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab71ef21b2040059fe683c26203cdf0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:51.831380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1166/2333] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:16:51 2022] DELETE /identity/v3/users/2ab71ef21b2040059fe683c26203cdf0 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:52.247217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:52.248957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:52.275690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2692fa8-4a11-4b78-b6aa-7bcce86c249e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:52.352246 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:52.352890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:52.353173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:52.353447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] PATH_INFO: `/v3/projects/93978cc4d1794e5d9c9419cbe70d680e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:52.354679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93978cc4d1794e5d9c9419cbe70d680e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:52.356594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:52.365692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:52.463917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93978cc4d1794e5d9c9419cbe70d680e', 'request_id': 'req-15691728-18df-4b1f-a8f7-29042e05e0b5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:52.474911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93978cc4d1794e5d9c9419cbe70d680e', 'request_id': 'req-15691728-18df-4b1f-a8f7-29042e05e0b5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:52.481479 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93978cc4d1794e5d9c9419cbe70d680e', 'request_id': 'req-15691728-18df-4b1f-a8f7-29042e05e0b5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:52.541921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] The token cache is being invalidate because project 93978cc4d1794e5d9c9419cbe70d680e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:16:52.543289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:52.543618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-15691728-18df-4b1f-a8f7-29042e05e0b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:52.546263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1168/2334] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:16:52 2022] DELETE /identity/v3/projects/93978cc4d1794e5d9c9419cbe70d680e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:52.550937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c26a5-beb4-4ad9-8143-032c7704863e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:52.551410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c26a5-beb4-4ad9-8143-032c7704863e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:52.551515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c26a5-beb4-4ad9-8143-032c7704863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:52.574221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-040c26a5-beb4-4ad9-8143-032c7704863e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:52.576936 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-040c26a5-beb4-4ad9-8143-032c7704863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:52.641042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1167/2335] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:52.649392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a8c6-ea8e-480b-a5c4-17b482f4e52c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:52.649786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a8c6-ea8e-480b-a5c4-17b482f4e52c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:52.650132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a8c6-ea8e-480b-a5c4-17b482f4e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:52.672066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-82b3a8c6-ea8e-480b-a5c4-17b482f4e52c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:52.673289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-82b3a8c6-ea8e-480b-a5c4-17b482f4e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:52.682176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1169/2336] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:52.686690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c26a5-beb4-4ad9-8143-032c7704863e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:52.697407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c26a5-beb4-4ad9-8143-032c7704863e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:52.702090 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:52.702676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:52.702956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:52.703243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:52.704012 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:16:52.704325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:52.706065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:52.719914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1168/2337] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:16:52.724591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3a8c6-ea8e-480b-a5c4-17b482f4e52c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:52.734295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3a8c6-ea8e-480b-a5c4-17b482f4e52c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:52.740195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:52.740760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:52.741046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:52.741321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:52.742051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:52.743905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:52.837658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1170/2338] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:16:52 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:16:52.841763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:52.853399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5ffad-24cb-47ee-968e-f02c8ee587fa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:52.921161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:52.921819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:52.922099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:52.922463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:52.923327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:52.926653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:53.026399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1169/2339] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:16:52 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:16:53.035004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:53.055410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a88e90-8c07-487d-9d1b-1f83f241b3aa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:53.061981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:53.063084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.063536 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.063663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.064519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:53.068080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:53.081599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1171/2340] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:16:53.082948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:53.093126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8d6d1-1435-40c6-be6a-06cc2730de27 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:53.099104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d52f2-1b61-498e-a6c2-b4eee8a891c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:53.099744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d52f2-1b61-498e-a6c2-b4eee8a891c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.100028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d52f2-1b61-498e-a6c2-b4eee8a891c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.100307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d52f2-1b61-498e-a6c2-b4eee8a891c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.100955 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d52f2-1b61-498e-a6c2-b4eee8a891c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:53.102697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d52f2-1b61-498e-a6c2-b4eee8a891c3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:53.115031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1170/2341] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:16:53.119362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:53.130673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9656-8b61-430d-918e-b6dfa2a8124f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:53.135526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:53.136185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.136502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.136774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] PATH_INFO: `/v3/projects/a53bbe1f55d64481807bbaa530585d5a/users/caf20ea7d6774eb386caddc6b7681df4/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.152327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53bbe1f55d64481807bbaa530585d5a, user_id=caf20ea7d6774eb386caddc6b7681df4, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:53.155118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:53.239768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1172/2342] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:16:53 2022] PUT /identity/v3/projects/a53bbe1f55d64481807bbaa530585d5a/users/caf20ea7d6774eb386caddc6b7681df4/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:53.252144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec014abe-eced-4315-8a56-b7568f698c31 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.252617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec014abe-eced-4315-8a56-b7568f698c31 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.253068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec014abe-eced-4315-8a56-b7568f698c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.295578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ec014abe-eced-4315-8a56-b7568f698c31 None None] MFA Rules not processed for user `caf20ea7d6774eb386caddc6b7681df4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:53.296942 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ec014abe-eced-4315-8a56-b7568f698c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:53.393509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:53.427275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1171/2343] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:53.431702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:53.511381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76696668-cd1b-4ca3-bd7f-a71cfa7abf00 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:53.517130 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:53.518649 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.518942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.519218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.519869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:53.522036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:53.644261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1173/2344] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:16:53 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:16:53.648154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec014abe-eced-4315-8a56-b7568f698c31 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:53.670008 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec014abe-eced-4315-8a56-b7568f698c31 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:53.734972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:53.735790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.736272 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.736618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.737327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:53.739484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:53.872691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1172/2345] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:16:53 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:16:53.874741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:53.888150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c77f-29b0-4fe9-b868-931f49bbe8ac admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:53.902098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:53.903000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.903487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.903942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.904998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:53.907040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:53.921645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1174/2346] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:16:53.924472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:53.935814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f96e1-4740-4b7c-a9c5-524ad04a6d77 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:53.941324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:53.942655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:53.942939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:53.943210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] PATH_INFO: `/v3/projects/4e446d35b3824cfc963e9dd7e5b45ebb/users/73c65059a0314231863515cb563d9f4e/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:53.961165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e446d35b3824cfc963e9dd7e5b45ebb, user_id=73c65059a0314231863515cb563d9f4e, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:53.964101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:54.058766 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1173/2347] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:16:53 2022] PUT /identity/v3/projects/4e446d35b3824cfc963e9dd7e5b45ebb/users/73c65059a0314231863515cb563d9f4e/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:54.062504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:54.103346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dadab92-9c5f-45d6-8c50-38383a1d0cdf admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:54.114742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:54.177457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c03687-466e-4f2a-8d09-5e5c9f7b2713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:54.178121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c03687-466e-4f2a-8d09-5e5c9f7b2713 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:54.178402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c03687-466e-4f2a-8d09-5e5c9f7b2713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:54.178675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c03687-466e-4f2a-8d09-5e5c9f7b2713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:54.179684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c03687-466e-4f2a-8d09-5e5c9f7b2713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:54.181011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c03687-466e-4f2a-8d09-5e5c9f7b2713 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:54.193829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1175/2348] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:16:54.197822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:54.212104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe683073-16cb-4e16-9942-7a5abde5f5db admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:54.221708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:54.222964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:54.223325 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:54.223668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] PATH_INFO: `/v3/projects/4e446d35b3824cfc963e9dd7e5b45ebb/users/73c65059a0314231863515cb563d9f4e/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:54.226663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e446d35b3824cfc963e9dd7e5b45ebb, user_id=73c65059a0314231863515cb563d9f4e, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:54.230152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:54.254949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1174/2349] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:16:54 2022] PUT /identity/v3/projects/4e446d35b3824cfc963e9dd7e5b45ebb/users/73c65059a0314231863515cb563d9f4e/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:54.266132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac08d-3933-4aa1-b20f-7e0fc33192c2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:54.266438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac08d-3933-4aa1-b20f-7e0fc33192c2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:54.266705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac08d-3933-4aa1-b20f-7e0fc33192c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:54.306733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ec7ac08d-3933-4aa1-b20f-7e0fc33192c2 None None] MFA Rules not processed for user `73c65059a0314231863515cb563d9f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:54.308172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ec7ac08d-3933-4aa1-b20f-7e0fc33192c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:54.414807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1176/2350] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:54.423400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:54.430832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:54.508966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdac7b-a299-4b54-bb0f-9fff5d0005b9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:54.514488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dd035-6fc0-45be-b892-eda449d96f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:54.517036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dd035-6fc0-45be-b892-eda449d96f6c service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:54.517214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dd035-6fc0-45be-b892-eda449d96f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:54.518823 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dd035-6fc0-45be-b892-eda449d96f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:54.530677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dd035-6fc0-45be-b892-eda449d96f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:54.535718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dd035-6fc0-45be-b892-eda449d96f6c service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:54.544859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1175/2351] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:55.194253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac08d-3933-4aa1-b20f-7e0fc33192c2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:55.281789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac08d-3933-4aa1-b20f-7e0fc33192c2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:55.289733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:55.290396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:55.290670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:55.290941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] PATH_INFO: `/v3/users/caf20ea7d6774eb386caddc6b7681df4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:55.292584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf20ea7d6774eb386caddc6b7681df4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:55.294834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:55.495747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:55.509849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf20ea7d6774eb386caddc6b7681df4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:55.510551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf20ea7d6774eb386caddc6b7681df4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:55.524898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf20ea7d6774eb386caddc6b7681df4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:55.627330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf20ea7d6774eb386caddc6b7681df4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:55.628066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf20ea7d6774eb386caddc6b7681df4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:55.643622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1177/2352] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:16:55 2022] DELETE /identity/v3/users/caf20ea7d6774eb386caddc6b7681df4 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:55.648515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:55.648993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:55.648993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:55.670044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:55.671661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:55.724517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:55.781993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1176/2353] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:55.786374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:55.789125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:55.860582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217dde-70e3-4dbc-908f-6f4ffad8cf7c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:55.865189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:55.866313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:55.866976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:55.868028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:55.881661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:55.883511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:55.902488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1178/2354] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:16:56.418257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:56.422157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:56.437409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbb5f5-f7b1-42c7-b9be-53a10d923ccc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:56.439029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:56.439029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:56.439896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:56.439896 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] PATH_INFO: `/v3/projects/a53bbe1f55d64481807bbaa530585d5a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:56.441216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a53bbe1f55d64481807bbaa530585d5a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:56.443236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:56.555551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a53bbe1f55d64481807bbaa530585d5a', 'request_id': 'req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:56.632218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a53bbe1f55d64481807bbaa530585d5a', 'request_id': 'req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:56.639050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a53bbe1f55d64481807bbaa530585d5a', 'request_id': 'req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:56.652496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] The token cache is being invalidate because project a53bbe1f55d64481807bbaa530585d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:16:56.653109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:56.653655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:56.655170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1177/2355] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:16:56 2022] DELETE /identity/v3/projects/a53bbe1f55d64481807bbaa530585d5a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:56.659793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:56.669865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9efbf5-b688-41a2-8eaf-e7d331c71c77 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:56.738808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:56.739527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:56.739884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:56.740254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] PATH_INFO: `/v3/users/73c65059a0314231863515cb563d9f4e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:56.742722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c65059a0314231863515cb563d9f4e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:56.744405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:56.981284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c65059a0314231863515cb563d9f4e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:56.982052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c65059a0314231863515cb563d9f4e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:56.987768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:56.998398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c65059a0314231863515cb563d9f4e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.072482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c65059a0314231863515cb563d9f4e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.073148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-16654553-e10d-4c85-a325-7af6e00d13e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c65059a0314231863515cb563d9f4e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.086717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1179/2356] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:16:56 2022] DELETE /identity/v3/users/73c65059a0314231863515cb563d9f4e => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:57.573933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:57.573933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:57.584102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c861-eb0b-4bf1-9ba3-6bab4c02c5b2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:57.661675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:57.662357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:57.662588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:57.662864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] PATH_INFO: `/v3/projects/4e446d35b3824cfc963e9dd7e5b45ebb` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:57.664728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e446d35b3824cfc963e9dd7e5b45ebb)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:57.667225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:57.777004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e446d35b3824cfc963e9dd7e5b45ebb', 'request_id': 'req-7518b2e0-73a3-42a5-9322-3c0bc36208c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.833962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e446d35b3824cfc963e9dd7e5b45ebb', 'request_id': 'req-7518b2e0-73a3-42a5-9322-3c0bc36208c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.839282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:57.841889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e446d35b3824cfc963e9dd7e5b45ebb', 'request_id': 'req-7518b2e0-73a3-42a5-9322-3c0bc36208c2'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.855136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] The token cache is being invalidate because project 4e446d35b3824cfc963e9dd7e5b45ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:16:57.855438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.856398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7518b2e0-73a3-42a5-9322-3c0bc36208c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:16:57.858270 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1178/2357] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:16:57 2022] DELETE /identity/v3/projects/4e446d35b3824cfc963e9dd7e5b45ebb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:16:57.863243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf8a0-f7bd-49b3-9ee0-db35d0693be5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:57.863719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf8a0-f7bd-49b3-9ee0-db35d0693be5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:57.863798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf8a0-f7bd-49b3-9ee0-db35d0693be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:57.904318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-03cdf8a0-f7bd-49b3-9ee0-db35d0693be5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:57.905632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03cdf8a0-f7bd-49b3-9ee0-db35d0693be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:57.976472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1180/2358] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:16:57.985342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0393a5-c097-4389-aafc-3e266b56e028 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:57.985681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0393a5-c097-4389-aafc-3e266b56e028 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:57.986551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0393a5-c097-4389-aafc-3e266b56e028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.008118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0e0393a5-c097-4389-aafc-3e266b56e028 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:58.009399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0e0393a5-c097-4389-aafc-3e266b56e028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:58.042014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1179/2359] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:16:58.044882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf8a0-f7bd-49b3-9ee0-db35d0693be5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.056915 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf8a0-f7bd-49b3-9ee0-db35d0693be5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:58.063233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0f19a-d3d5-464d-a72d-ed1310261802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:58.077293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f19a-d3d5-464d-a72d-ed1310261802 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.077293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f19a-d3d5-464d-a72d-ed1310261802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.077293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f19a-d3d5-464d-a72d-ed1310261802 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.077293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f19a-d3d5-464d-a72d-ed1310261802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:16:58.077293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f19a-d3d5-464d-a72d-ed1310261802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:58.077293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f19a-d3d5-464d-a72d-ed1310261802 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:58.085128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1181/2360] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:16:58.092251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0393a5-c097-4389-aafc-3e266b56e028 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.103855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0393a5-c097-4389-aafc-3e266b56e028 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:58.107713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.108004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.110626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.113622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:58.114522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.114998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.115267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.118451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:58.120458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:58.131513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:58.132787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:58.196413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1182/2361] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:16:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:16:58.210757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.212463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:58.242523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1180/2362] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:16:58 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:16:58.248148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.311443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:16:58.326137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4a181-015e-4249-a9c5-086713037159 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:58.340032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26f16b-4eaa-4a74-927a-90e99526d180 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:58.347441 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:58.348065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.348339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.349023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.349677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:58.352339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:58.419562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:58.420413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.420952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.421311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.467106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1181/2363] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:16:58 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:16:58.474035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.489974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009738e-9677-4d47-9fbe-651912101df9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:58.496690 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:58.497496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.498035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.498268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.498721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:58.500740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:58.518773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1182/2364] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:16:58.522795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.529224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:58.530964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:58.537440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ed4e8-8050-469c-8d11-ea2b93224fe6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:58.551534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff5ed6-6544-4c64-af22-81436412523c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:58.551534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5ed6-6544-4c64-af22-81436412523c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.551534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5ed6-6544-4c64-af22-81436412523c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.551534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5ed6-6544-4c64-af22-81436412523c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.551903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff5ed6-6544-4c64-af22-81436412523c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:58.551980 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1183/2365] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:16:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 04 16:16:58.558399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff5ed6-6544-4c64-af22-81436412523c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:58.575105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1183/2366] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:16:58.576011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.587962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d8eaf-d570-496f-8ce3-040de248eb37 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:58.593093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:58.593993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.594290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.594574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] PATH_INFO: `/v3/projects/e99fa65162774d7b9ec953813a2326b8/users/33264517272a4af88ff7b9232e3207c4/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.616465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e99fa65162774d7b9ec953813a2326b8, user_id=33264517272a4af88ff7b9232e3207c4, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:58.620026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:58.785160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1184/2367] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:16:58 2022] PUT /identity/v3/projects/e99fa65162774d7b9ec953813a2326b8/users/33264517272a4af88ff7b9232e3207c4/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:16:58.795486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:58.796081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:58.796484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:58.837112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] MFA Rules not processed for user `33264517272a4af88ff7b9232e3207c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:16:58.838457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:58.945135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1184/2368] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:16:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:16:58.970073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:16:58.973325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:16:59.074896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3c6e5-3192-4ee9-9f35-b556fd15813d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:16:59.080343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:16:59.081837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:16:59.082220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:16:59.084293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:16:59.097674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:16:59.099378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:16:59.128544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1185/2369] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:16:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:08.940591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=1,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:09.457263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:12.645185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:12.646705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:12.702017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeceee-de57-43a4-81d9-b37cae18c20d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:12.705648 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:12.706243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:12.706519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:12.706787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:12.708528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:12.776830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:12.778289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:12.788694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1185/2370] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:16.245816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:16.255800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2aa3-02d3-4e7c-a683-209badb59940 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:16.259364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b97694-a162-4c27-ac78-ed09a6c9b771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:16.259990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b97694-a162-4c27-ac78-ed09a6c9b771 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:16.260267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b97694-a162-4c27-ac78-ed09a6c9b771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:16.260535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b97694-a162-4c27-ac78-ed09a6c9b771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:16.262279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f3b97694-a162-4c27-ac78-ed09a6c9b771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:16.324328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b97694-a162-4c27-ac78-ed09a6c9b771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:16.325813 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b97694-a162-4c27-ac78-ed09a6c9b771 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:16.336168 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1186/2371] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:22.783560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=50 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:24.765194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:24.826827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa318-a3b7-49bb-ad8f-c58e078eed79 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:24.831411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:24.832037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:24.832354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:24.832651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] PATH_INFO: `/v3/users/33264517272a4af88ff7b9232e3207c4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:24.833881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33264517272a4af88ff7b9232e3207c4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:24.835800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:25.011627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33264517272a4af88ff7b9232e3207c4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.012311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33264517272a4af88ff7b9232e3207c4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.059838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33264517272a4af88ff7b9232e3207c4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.060459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33264517272a4af88ff7b9232e3207c4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.073460 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33264517272a4af88ff7b9232e3207c4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.086091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1186/2372] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:17:24 2022] DELETE /identity/v3/users/33264517272a4af88ff7b9232e3207c4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:25.090653 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:25.091122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:25.091229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:25.111198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:25.112400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:25.171639 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1187/2373] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:25.181174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:25.182628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:25.239403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32aa27-f6ea-45e5-b12e-a25b0ececcc8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:25.242980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65995c-e8d2-42da-9ac4-0d0c2e278645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:25.243571 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65995c-e8d2-42da-9ac4-0d0c2e278645 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:25.243853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65995c-e8d2-42da-9ac4-0d0c2e278645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:25.244123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65995c-e8d2-42da-9ac4-0d0c2e278645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:25.252908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65995c-e8d2-42da-9ac4-0d0c2e278645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:25.254481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65995c-e8d2-42da-9ac4-0d0c2e278645 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:25.264422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1187/2374] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:25.627212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:25.628872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:25.643573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96c219-f5a4-45c0-92bb-7684d97d7b09 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:25.648539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:25.655127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:25.655127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:25.655127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] PATH_INFO: `/v3/projects/e99fa65162774d7b9ec953813a2326b8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:25.655127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e99fa65162774d7b9ec953813a2326b8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:25.655127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:25.758360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e99fa65162774d7b9ec953813a2326b8', 'request_id': 'req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.764249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e99fa65162774d7b9ec953813a2326b8', 'request_id': 'req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.771019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e99fa65162774d7b9ec953813a2326b8', 'request_id': 'req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.844093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] The token cache is being invalidate because project e99fa65162774d7b9ec953813a2326b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:17:25.852584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.852584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-062c16b1-4630-4cc2-84cf-c7adac9a5ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:25.852584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1188/2375] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:17:25 2022] DELETE /identity/v3/projects/e99fa65162774d7b9ec953813a2326b8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:25.852839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737a835-5471-420e-8378-37293e2f621c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:25.852839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737a835-5471-420e-8378-37293e2f621c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:25.852839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737a835-5471-420e-8378-37293e2f621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:25.873015 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3737a835-5471-420e-8378-37293e2f621c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:25.874257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3737a835-5471-420e-8378-37293e2f621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:25.893191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:25.937616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1188/2376] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:25.948811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf53831-130b-481b-a019-097307c2809b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:25.949092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf53831-130b-481b-a019-097307c2809b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:25.949377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf53831-130b-481b-a019-097307c2809b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:25.971304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-edf53831-130b-481b-a019-097307c2809b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:25.972525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-edf53831-130b-481b-a019-097307c2809b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:25.981770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1189/2377] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:25.986025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737a835-5471-420e-8378-37293e2f621c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:25.996605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737a835-5471-420e-8378-37293e2f621c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:26.001426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:26.002085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.002375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.002695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.003296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:17:26.003599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:26.005355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:26.017188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1189/2378] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:17:26.022700 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf53831-130b-481b-a019-097307c2809b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:26.031892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf53831-130b-481b-a019-097307c2809b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:26.037069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:26.037693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.037970 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.038242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.038875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:26.040773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:26.050197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:26.107317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1190/2379] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:17:26 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:17:26.111533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:26.121923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f09720-cd88-4498-8d85-8b7184756c6a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:26.179187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:26.179812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.180093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.180366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.181000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:26.182952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:26.254050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1190/2380] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:17:26 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:17:26.258604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:26.269078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561584c7-2fe7-4dd5-9ac1-16466b5e209d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:26.274624 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:26.275212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.275491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.275761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.276390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:26.278021 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:26.287834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1191/2381] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:17:26.293048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:26.302887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430e9c-7f69-4dd3-8733-5b8f8ec59743 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:26.307497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffd97f-fa94-41b5-97c5-aa77b285ad24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:26.308085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffd97f-fa94-41b5-97c5-aa77b285ad24 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.308366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffd97f-fa94-41b5-97c5-aa77b285ad24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.308641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffd97f-fa94-41b5-97c5-aa77b285ad24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.309278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffd97f-fa94-41b5-97c5-aa77b285ad24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:26.310904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffd97f-fa94-41b5-97c5-aa77b285ad24 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:26.322414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1191/2382] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:17:26.326315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:26.339110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48b3dc-9fec-4cab-85ec-c917bad94356 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:26.343869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:26.344520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.344797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.345240 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] PATH_INFO: `/v3/projects/8f47ddf92da64329938e7989e2b5d3c0/users/79a20c93acba49029130d241c9ce7ef3/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.359888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f47ddf92da64329938e7989e2b5d3c0, user_id=79a20c93acba49029130d241c9ce7ef3, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:26.362633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:26.419726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1192/2383] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:17:26 2022] PUT /identity/v3/projects/8f47ddf92da64329938e7989e2b5d3c0/users/79a20c93acba49029130d241c9ce7ef3/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:26.431162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b406bce-6893-4240-a8cb-69a199e10fee None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.431477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b406bce-6893-4240-a8cb-69a199e10fee None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.431779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b406bce-6893-4240-a8cb-69a199e10fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.459278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0b406bce-6893-4240-a8cb-69a199e10fee None None] MFA Rules not processed for user `79a20c93acba49029130d241c9ce7ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:26.460470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0b406bce-6893-4240-a8cb-69a199e10fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:26.524201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1192/2384] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:26.534684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:26.536935 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:26.592284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8cba1-aa10-4e68-8697-d914a5ca2528 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:26.595913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:26.596503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:26.596780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:26.597048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:26.605891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:26.607316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:26.616963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1193/2385] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:27.631852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed66f8d-062e-4065-b976-a5a1231a64ad None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:27.632298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed66f8d-062e-4065-b976-a5a1231a64ad None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:27.632489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed66f8d-062e-4065-b976-a5a1231a64ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:27.653485 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3ed66f8d-062e-4065-b976-a5a1231a64ad None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:27.654715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3ed66f8d-062e-4065-b976-a5a1231a64ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:27.672341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:27.715834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1193/2386] 10.222.0.39 () {58 vars in 967 bytes} [Sun Sep 4 16:17:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:27.738267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:27.739073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:27.808816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a59fcd-5b7c-4e17-8685-c47de514482d service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:27.813925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e89f53-1a86-42f8-aa5b-3784a9d818d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:27.814530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e89f53-1a86-42f8-aa5b-3784a9d818d1 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:27.814869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e89f53-1a86-42f8-aa5b-3784a9d818d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:27.815165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e89f53-1a86-42f8-aa5b-3784a9d818d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:27.824205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e89f53-1a86-42f8-aa5b-3784a9d818d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:27.825825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e89f53-1a86-42f8-aa5b-3784a9d818d1 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:27.834521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1194/2387] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:17:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:17:30.490768 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed66f8d-062e-4065-b976-a5a1231a64ad None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:30.552442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed66f8d-062e-4065-b976-a5a1231a64ad None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:30.557028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:30.557685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:30.557958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:30.558253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] PATH_INFO: `/v3/users/79a20c93acba49029130d241c9ce7ef3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:30.559365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a20c93acba49029130d241c9ce7ef3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:30.561291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:30.759395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a20c93acba49029130d241c9ce7ef3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:30.759966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a20c93acba49029130d241c9ce7ef3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:30.813733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a20c93acba49029130d241c9ce7ef3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:30.814360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a20c93acba49029130d241c9ce7ef3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:30.827260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a20c93acba49029130d241c9ce7ef3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:30.839355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1194/2388] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:17:30 2022] DELETE /identity/v3/users/79a20c93acba49029130d241c9ce7ef3 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:30.843847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:30.844315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:30.844424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:30.864273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:30.865457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:30.911223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:30.930177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1195/2389] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:30.939806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:30.941210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:30.987275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:30.998899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d5ab2-21ed-4928-95a5-5fb4431782cd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:31.002576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711226bb-244b-448d-baff-e5546ec81c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:31.003172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711226bb-244b-448d-baff-e5546ec81c4f service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:31.003449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711226bb-244b-448d-baff-e5546ec81c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:31.003727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711226bb-244b-448d-baff-e5546ec81c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:31.021291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711226bb-244b-448d-baff-e5546ec81c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:31.021291 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711226bb-244b-448d-baff-e5546ec81c4f service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:31.026594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1195/2390] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:31.438494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:31.440179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:31.453663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8013-e0dc-4103-b5f7-0309248325a4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:31.458982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:31.459625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:31.459913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:31.460189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] PATH_INFO: `/v3/projects/8f47ddf92da64329938e7989e2b5d3c0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:31.461425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f47ddf92da64329938e7989e2b5d3c0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:31.463356 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:31.562388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f47ddf92da64329938e7989e2b5d3c0', 'request_id': 'req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:31.568247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f47ddf92da64329938e7989e2b5d3c0', 'request_id': 'req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:31.574299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f47ddf92da64329938e7989e2b5d3c0', 'request_id': 'req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:31.665910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] The token cache is being invalidate because project 8f47ddf92da64329938e7989e2b5d3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:17:31.666205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:31.666848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-263fa9c2-ee5a-4dd9-a6bd-75300e24d7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:31.668334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1196/2391] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:17:31 2022] DELETE /identity/v3/projects/8f47ddf92da64329938e7989e2b5d3c0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:31.673310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12589326-41eb-4e28-927b-659b04da6de5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:31.673788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12589326-41eb-4e28-927b-659b04da6de5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:31.677033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12589326-41eb-4e28-927b-659b04da6de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:31.693725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-12589326-41eb-4e28-927b-659b04da6de5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:31.694908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-12589326-41eb-4e28-927b-659b04da6de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:31.752550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1196/2392] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:31.761228 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30865b9-b8f0-4b58-98d9-b2c66bf1e416 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:31.761522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30865b9-b8f0-4b58-98d9-b2c66bf1e416 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:31.761835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30865b9-b8f0-4b58-98d9-b2c66bf1e416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:31.781881 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c30865b9-b8f0-4b58-98d9-b2c66bf1e416 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:31.783084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c30865b9-b8f0-4b58-98d9-b2c66bf1e416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:31.792031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1197/2393] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:31.796168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12589326-41eb-4e28-927b-659b04da6de5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:31.806417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12589326-41eb-4e28-927b-659b04da6de5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:31.810995 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:31.811610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:31.811888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:31.812159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:31.812849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:17:31.813102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:31.814774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:31.826636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1197/2394] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:17:31.830782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30865b9-b8f0-4b58-98d9-b2c66bf1e416 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:31.840409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30865b9-b8f0-4b58-98d9-b2c66bf1e416 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:31.844993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:31.845603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:31.845884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:31.846153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:31.846782 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:31.849519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:31.913291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1198/2395] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:17:31 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:17:31.916727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:31.927449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78c13a-b297-4644-917f-6dea31a3ed4a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:31.990495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:31.991098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:31.991378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:31.991651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:31.992326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:31.994262 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:32.077774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1198/2396] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:17:31 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:17:32.081089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:32.091348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c2491-54ae-4346-9e51-f315f12b4f14 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:32.096000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:32.096636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:32.096911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:32.097182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:32.097858 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:32.100625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:32.109417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1199/2397] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:17:32.113274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:32.124081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683446-fb31-4694-9e6b-422104dfc8c9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:32.128719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dfd3f-507d-4a5c-815d-a2789b7296e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:32.130022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dfd3f-507d-4a5c-815d-a2789b7296e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:32.130265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dfd3f-507d-4a5c-815d-a2789b7296e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:32.130541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dfd3f-507d-4a5c-815d-a2789b7296e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:32.131183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dfd3f-507d-4a5c-815d-a2789b7296e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:32.132770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dfd3f-507d-4a5c-815d-a2789b7296e5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:32.143430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1199/2398] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:17:32.147457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:32.157227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55040d8a-0b8e-41f8-90c7-256704ea05b5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:32.161997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:32.162629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:32.162908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:32.163178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] PATH_INFO: `/v3/projects/f66ef32d823c4cd5bed66be7b4b5f2b5/users/f4720c4916084dc0a9696b21ecd28a2c/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:32.178845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66ef32d823c4cd5bed66be7b4b5f2b5, user_id=f4720c4916084dc0a9696b21ecd28a2c, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:32.181487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:32.237276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1200/2399] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:17:32 2022] PUT /identity/v3/projects/f66ef32d823c4cd5bed66be7b4b5f2b5/users/f4720c4916084dc0a9696b21ecd28a2c/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:32.249033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83645c71-1bbd-42ba-8653-d7792429f07f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:32.249317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83645c71-1bbd-42ba-8653-d7792429f07f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:32.249619 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83645c71-1bbd-42ba-8653-d7792429f07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:32.276576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-83645c71-1bbd-42ba-8653-d7792429f07f None None] MFA Rules not processed for user `f4720c4916084dc0a9696b21ecd28a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:32.277806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-83645c71-1bbd-42ba-8653-d7792429f07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:32.317431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:32.341558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1200/2400] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:32.353589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:32.357767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:32.411048 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5a21b-89f3-488b-9961-3c05a9b04eee admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:32.414894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e0963-69da-4b47-9bc5-172120b7b6d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:32.415504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e0963-69da-4b47-9bc5-172120b7b6d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:32.415901 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e0963-69da-4b47-9bc5-172120b7b6d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:32.416188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e0963-69da-4b47-9bc5-172120b7b6d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:32.424541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e0963-69da-4b47-9bc5-172120b7b6d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:32.426704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e0963-69da-4b47-9bc5-172120b7b6d9 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:32.435411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1201/2401] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:35.465645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83645c71-1bbd-42ba-8653-d7792429f07f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:35.526587 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83645c71-1bbd-42ba-8653-d7792429f07f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:35.531029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:35.531651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:35.532038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:35.532344 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] PATH_INFO: `/v3/users/f4720c4916084dc0a9696b21ecd28a2c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:35.533470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4720c4916084dc0a9696b21ecd28a2c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:35.535439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:35.685872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4720c4916084dc0a9696b21ecd28a2c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:35.686521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4720c4916084dc0a9696b21ecd28a2c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:35.728190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4720c4916084dc0a9696b21ecd28a2c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:35.728814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4720c4916084dc0a9696b21ecd28a2c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:35.741558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4720c4916084dc0a9696b21ecd28a2c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:35.753533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1201/2402] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:17:35 2022] DELETE /identity/v3/users/f4720c4916084dc0a9696b21ecd28a2c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:17:35.758599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:35.759066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:35.759181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:35.778287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:35.779469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:35.783439 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:35.838691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1202/2403] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:35.848280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:35.849933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:35.908767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fbe0d-33e1-4992-912a-849afa35b2a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:35.913081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030ecc6-a90f-412c-a83f-49b859e81c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:35.913729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ecc6-a90f-412c-a83f-49b859e81c9b service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:35.914009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ecc6-a90f-412c-a83f-49b859e81c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:35.914280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ecc6-a90f-412c-a83f-49b859e81c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:35.923260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030ecc6-a90f-412c-a83f-49b859e81c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:35.924957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030ecc6-a90f-412c-a83f-49b859e81c9b service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:35.935699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1202/2404] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:36.295566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:36.297176 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:36.310040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550d76e-856e-4390-a687-740b02bbe02b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:36.315124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:36.315771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:36.316055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:36.316333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] PATH_INFO: `/v3/projects/f66ef32d823c4cd5bed66be7b4b5f2b5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:36.317832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66ef32d823c4cd5bed66be7b4b5f2b5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:36.319965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:36.420487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66ef32d823c4cd5bed66be7b4b5f2b5', 'request_id': 'req-90b0c777-9063-4e86-86f0-5cca0e69418f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:36.425867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66ef32d823c4cd5bed66be7b4b5f2b5', 'request_id': 'req-90b0c777-9063-4e86-86f0-5cca0e69418f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:36.432513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66ef32d823c4cd5bed66be7b4b5f2b5', 'request_id': 'req-90b0c777-9063-4e86-86f0-5cca0e69418f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:36.505673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] The token cache is being invalidate because project f66ef32d823c4cd5bed66be7b4b5f2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:17:36.506013 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:36.506638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-90b0c777-9063-4e86-86f0-5cca0e69418f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:17:36.508119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1203/2405] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:17:36 2022] DELETE /identity/v3/projects/f66ef32d823c4cd5bed66be7b4b5f2b5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:36.513054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7f87-8692-4907-9670-db63bcba55ae None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:36.513509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7f87-8692-4907-9670-db63bcba55ae None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:36.513643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7f87-8692-4907-9670-db63bcba55ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:36.521824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:36.534024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-da7c7f87-8692-4907-9670-db63bcba55ae None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:36.535239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-da7c7f87-8692-4907-9670-db63bcba55ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:36.596233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1203/2406] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:36.604359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0525dc-5d20-4e23-a251-e0da07b4cd74 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:36.604643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0525dc-5d20-4e23-a251-e0da07b4cd74 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:36.604920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0525dc-5d20-4e23-a251-e0da07b4cd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:36.625809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3b0525dc-5d20-4e23-a251-e0da07b4cd74 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:36.627180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3b0525dc-5d20-4e23-a251-e0da07b4cd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:36.636309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1204/2407] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:36.640783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c7f87-8692-4907-9670-db63bcba55ae None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:36.650825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c7f87-8692-4907-9670-db63bcba55ae None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:36.655394 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:36.655986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:36.656265 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:36.656537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:36.657175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:17:36.657474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:36.659080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:36.677323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1204/2408] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:17:36.682006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0525dc-5d20-4e23-a251-e0da07b4cd74 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:36.691291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0525dc-5d20-4e23-a251-e0da07b4cd74 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:36.696866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:36.697459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:36.697774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:36.698049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:36.698683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:36.700573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:36.786450 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1205/2409] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:17:36 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:17:36.790636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:36.800592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d14065-f56c-4773-8d09-10a6e4bffade admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:36.857990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:36.858599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:36.859020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:36.859369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:36.860042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:36.862004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:36.962736 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1205/2410] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:17:36 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:17:36.966824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:36.980973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e95cd-e0fa-4592-8a2c-5a42017ac541 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:36.987038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:36.987631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:36.987910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:36.988191 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:36.988845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:36.991049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:37.004183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1206/2411] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:17:37.005621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:37.015769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c399e9-8b79-4921-aeac-8baea0ff8e2f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:37.020467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c87b0-3df3-423d-985b-46230146578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:37.021053 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c87b0-3df3-423d-985b-46230146578d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:37.021332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c87b0-3df3-423d-985b-46230146578d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:37.021634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c87b0-3df3-423d-985b-46230146578d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:37.022269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c87b0-3df3-423d-985b-46230146578d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:37.023846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c87b0-3df3-423d-985b-46230146578d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:37.034238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1206/2412] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:17:37.039215 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:37.049175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266850d-edfc-4730-a3bf-807fe233b562 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:37.054490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:37.055134 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:37.055416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:37.055691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] PATH_INFO: `/v3/projects/d60428e7a1454bc9af57e305abcff0eb/users/bf3e97f2ef674d2bb36d5094a707664f/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:37.071120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60428e7a1454bc9af57e305abcff0eb, user_id=bf3e97f2ef674d2bb36d5094a707664f, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:37.073815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:37.127804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1207/2413] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:17:37 2022] PUT /identity/v3/projects/d60428e7a1454bc9af57e305abcff0eb/users/bf3e97f2ef674d2bb36d5094a707664f/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:17:37.138417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7afda8-60ea-43b6-9b96-d4159ee56d8a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:37.138709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7afda8-60ea-43b6-9b96-d4159ee56d8a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:37.139252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7afda8-60ea-43b6-9b96-d4159ee56d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:37.174075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8e7afda8-60ea-43b6-9b96-d4159ee56d8a None None] MFA Rules not processed for user `bf3e97f2ef674d2bb36d5094a707664f`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:17:37.175321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8e7afda8-60ea-43b6-9b96-d4159ee56d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:37.252615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:37.258315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1207/2414] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:17:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:17:37.270111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:17:37.272296 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:17:37.304415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:37.331715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37b9c6-63c1-40ca-b000-b71d581389ad admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:17:37.336042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa402b-e13c-488b-a26b-c3a4f18ec935 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95204bd338a4a99a510a67c4ab97532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:17:37.336544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa402b-e13c-488b-a26b-c3a4f18ec935 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:17:37.336877 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa402b-e13c-488b-a26b-c3a4f18ec935 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:17:37.337144 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa402b-e13c-488b-a26b-c3a4f18ec935 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:17:37.345964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa402b-e13c-488b-a26b-c3a4f18ec935 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:17:37.347423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa402b-e13c-488b-a26b-c3a4f18ec935 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:17:37.356997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1208/2415] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:17:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:17:47.299197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:17:47.373878 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=13 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:15.329249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7afda8-60ea-43b6-9b96-d4159ee56d8a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:15.399760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7afda8-60ea-43b6-9b96-d4159ee56d8a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:15.404414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:15.405056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:15.405334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:15.405637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] PATH_INFO: `/v3/users/bf3e97f2ef674d2bb36d5094a707664f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:15.406770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3e97f2ef674d2bb36d5094a707664f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:15.408719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:15.611280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3e97f2ef674d2bb36d5094a707664f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:15.611969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3e97f2ef674d2bb36d5094a707664f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:15.678193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3e97f2ef674d2bb36d5094a707664f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:15.679649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3e97f2ef674d2bb36d5094a707664f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:15.693184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3e97f2ef674d2bb36d5094a707664f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:15.707496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1208/2416] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:18:15 2022] DELETE /identity/v3/users/bf3e97f2ef674d2bb36d5094a707664f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:18:15.712251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:15.712752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:15.712862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:15.731966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:15.733174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:15.791907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1209/2417] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:15.801664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:15.803141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:15.860624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da964f60-7bbc-42ff-8601-54e3ce723c3c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:15.864279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:15.864875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:15.865151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:15.865420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:15.874406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:15.875809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:15.885440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1209/2418] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:18:16.302306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:16.303756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:16.317891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6a6bf-c6fa-4205-b841-98568f45cc34 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:16.323735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:16.324370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:16.324651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:16.324994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] PATH_INFO: `/v3/projects/d60428e7a1454bc9af57e305abcff0eb` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:16.327438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60428e7a1454bc9af57e305abcff0eb)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:16.329514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:16.438596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60428e7a1454bc9af57e305abcff0eb', 'request_id': 'req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:16.444287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60428e7a1454bc9af57e305abcff0eb', 'request_id': 'req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:16.452529 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60428e7a1454bc9af57e305abcff0eb', 'request_id': 'req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:16.517398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] The token cache is being invalidate because project d60428e7a1454bc9af57e305abcff0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:18:16.517617 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:16.518232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:16.519698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1210/2419] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:18:16 2022] DELETE /identity/v3/projects/d60428e7a1454bc9af57e305abcff0eb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:18:18.108979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:18.113039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:18.124799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337c0dc-d357-4cb6-91bf-06ad532767c4 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:18.142880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:18.180653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:18.181271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:18.181547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:18.181850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] PATH_INFO: `/v3/users/a8925da59209424fbd6375f056aee752` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:18.182966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8925da59209424fbd6375f056aee752)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:18.184869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:18.327254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8925da59209424fbd6375f056aee752'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.327903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8925da59209424fbd6375f056aee752'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.420549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8925da59209424fbd6375f056aee752'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.421217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8925da59209424fbd6375f056aee752'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.433711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-156fa00d-79d1-4163-b2a4-64326aca57ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8925da59209424fbd6375f056aee752'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.445546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1210/2420] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:18:18 2022] DELETE /identity/v3/users/a8925da59209424fbd6375f056aee752 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:18:18.690726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:18.692173 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:18.706713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8277bb-6a1e-4788-8fac-8f4c3fd442e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:18.766496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:18.767137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:18.767426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:18.767702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] PATH_INFO: `/v3/projects/9f1ae25e634d4380a6933101fb94ad2e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:18.768818 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1ae25e634d4380a6933101fb94ad2e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:18.770787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:18.879844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:18.898562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1ae25e634d4380a6933101fb94ad2e', 'request_id': 'req-c24710f1-00f1-452f-a409-3919fb9df7e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.904418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1ae25e634d4380a6933101fb94ad2e', 'request_id': 'req-c24710f1-00f1-452f-a409-3919fb9df7e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.910551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1ae25e634d4380a6933101fb94ad2e', 'request_id': 'req-c24710f1-00f1-452f-a409-3919fb9df7e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.997288 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] The token cache is being invalidate because project 9f1ae25e634d4380a6933101fb94ad2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:18:18.997598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.998229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c24710f1-00f1-452f-a409-3919fb9df7e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:18:18.999741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1211/2421] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:18:18 2022] DELETE /identity/v3/projects/9f1ae25e634d4380a6933101fb94ad2e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:18:19.004699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6f3f-5ef8-45bc-9f23-c889af974b25 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.005073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6f3f-5ef8-45bc-9f23-c889af974b25 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.005418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6f3f-5ef8-45bc-9f23-c889af974b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.026474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-12ed6f3f-5ef8-45bc-9f23-c889af974b25 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:19.027664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-12ed6f3f-5ef8-45bc-9f23-c889af974b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:19.084696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1211/2422] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:19.093493 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88beeb9-1b58-4068-81aa-4f038ac5c66b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.093810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88beeb9-1b58-4068-81aa-4f038ac5c66b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.094083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88beeb9-1b58-4068-81aa-4f038ac5c66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.113357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d88beeb9-1b58-4068-81aa-4f038ac5c66b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:19.114595 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d88beeb9-1b58-4068-81aa-4f038ac5c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:19.122524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1212/2423] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:19.126501 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed6f3f-5ef8-45bc-9f23-c889af974b25 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:19.137411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed6f3f-5ef8-45bc-9f23-c889af974b25 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:19.142355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:19.142954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.143236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.143509 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.144154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:18:19.144459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:19.146077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:19.158259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1212/2424] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:18:19.162461 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88beeb9-1b58-4068-81aa-4f038ac5c66b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:19.171993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88beeb9-1b58-4068-81aa-4f038ac5c66b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:19.176676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:19.177271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.177554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.177859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.178495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:19.181164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:19.267003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1213/2425] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:18:19 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:18:19.271158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:19.281168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd358a-0ee3-4cfa-a923-71b71b9027ea admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:19.306984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:19.338625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:19.339123 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.339645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.340261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.341237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:19.343347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:19.461540 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1213/2426] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:18:19 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:18:19.465383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:19.475549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939bba0-92c3-4bea-a69a-634d16bc6b1a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:19.480146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:19.480958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.481250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.481607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.482248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:19.483856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:19.494260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1214/2427] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:18:19.498215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:19.508861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3097e-c484-4458-b310-e613eb2e70c2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:19.513499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a7412-34bc-4ac3-b060-a1a14f446fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:19.514125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a7412-34bc-4ac3-b060-a1a14f446fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.514402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a7412-34bc-4ac3-b060-a1a14f446fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.514673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a7412-34bc-4ac3-b060-a1a14f446fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.515318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a7412-34bc-4ac3-b060-a1a14f446fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:19.517145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a7412-34bc-4ac3-b060-a1a14f446fdf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:19.527172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1214/2428] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:18:19.531281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:19.542410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b01f5a-2212-4d3d-95e5-540d43217ab7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:19.547098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:19.547732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.548008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.548308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] PATH_INFO: `/v3/projects/ae092c3124e9438fa20dfda624775825/users/9b17d0d51b034064a98bdae4bbc20fb6/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.563738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae092c3124e9438fa20dfda624775825, user_id=9b17d0d51b034064a98bdae4bbc20fb6, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:19.566522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:19.632899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1215/2429] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:18:19 2022] PUT /identity/v3/projects/ae092c3124e9438fa20dfda624775825/users/9b17d0d51b034064a98bdae4bbc20fb6/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:18:19.638009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adf91-6f1e-4c30-8109-6b0c9541270f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.638322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adf91-6f1e-4c30-8109-6b0c9541270f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.638661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adf91-6f1e-4c30-8109-6b0c9541270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.658317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-522adf91-6f1e-4c30-8109-6b0c9541270f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:19.659636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-522adf91-6f1e-4c30-8109-6b0c9541270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:19.720552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1215/2430] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:19.730796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:19.732582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:19.790453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41dc42-7fb1-49ea-b397-e84ab6c8fa69 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:19.794116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:19.794715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:19.794999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:19.795277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:19.804430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:19.806670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:19.816285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1216/2431] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:18:24.823757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157442e-3d67-43ab-b319-029c8f037ced None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:24.824728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157442e-3d67-43ab-b319-029c8f037ced None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:24.825327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157442e-3d67-43ab-b319-029c8f037ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:24.870157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1157442e-3d67-43ab-b319-029c8f037ced None None] MFA Rules not processed for user `9b17d0d51b034064a98bdae4bbc20fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:24.871389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1157442e-3d67-43ab-b319-029c8f037ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:24.946671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1216/2432] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:24.951285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:24.962289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c43759-a06b-44ae-9fda-c30c9f441e9f service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:24.967117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:24.967734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:24.968019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:24.968300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:24.968952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:24.972079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:25.051506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1217/2433] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:18:24 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:18:25.056384 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157442e-3d67-43ab-b319-029c8f037ced None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:25.068769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157442e-3d67-43ab-b319-029c8f037ced None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:25.105117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:25.129835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:25.130446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:25.130731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:25.131006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:25.131640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:25.133898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:25.246140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1217/2434] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:18:25 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:18:25.249118 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:25.261718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fbfc0-7a4f-4398-b4db-5f67da7b461f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:25.266066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:25.267150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:25.267435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:25.267712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:25.268368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:25.270014 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:25.282152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1218/2435] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:18:25.286406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:25.297750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaf34-e650-4be4-8e47-f36b57a4f401 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:25.302435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:25.303099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:25.303379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:25.303651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] PATH_INFO: `/v3/projects/9e40d38756e04664a630bfc16c9d411d/users/7e6f70cd24b7424ab0cf94c494ebdcbc/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:25.323832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e40d38756e04664a630bfc16c9d411d, user_id=7e6f70cd24b7424ab0cf94c494ebdcbc, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:25.326593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:25.393364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1218/2436] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:18:25 2022] PUT /identity/v3/projects/9e40d38756e04664a630bfc16c9d411d/users/7e6f70cd24b7424ab0cf94c494ebdcbc/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:18:25.397513 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:25.408361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b82c21-85c3-48d8-b07f-5c64cc26c3ba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:25.427979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:25.469111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f66619-e16b-44b6-9ea0-ee989219cdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:25.469766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66619-e16b-44b6-9ea0-ee989219cdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:25.470051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66619-e16b-44b6-9ea0-ee989219cdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:25.470329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66619-e16b-44b6-9ea0-ee989219cdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:25.470969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f66619-e16b-44b6-9ea0-ee989219cdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:25.472590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f66619-e16b-44b6-9ea0-ee989219cdf2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:25.482790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1219/2437] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:18:25.486802 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:25.497534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab4acc-fa47-45a5-a1dc-841ab977ccdc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:25.502479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:25.503143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:25.503428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:25.503703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] PATH_INFO: `/v3/projects/9e40d38756e04664a630bfc16c9d411d/users/7e6f70cd24b7424ab0cf94c494ebdcbc/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:25.505697 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e40d38756e04664a630bfc16c9d411d, user_id=7e6f70cd24b7424ab0cf94c494ebdcbc, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:25.508548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:25.519369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1219/2438] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:18:25 2022] PUT /identity/v3/projects/9e40d38756e04664a630bfc16c9d411d/users/7e6f70cd24b7424ab0cf94c494ebdcbc/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:18:32.036421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5964b34-70ec-4ccc-ae45-e0c20fd85593 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:32.036941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5964b34-70ec-4ccc-ae45-e0c20fd85593 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:32.037090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5964b34-70ec-4ccc-ae45-e0c20fd85593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:32.067484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c5964b34-70ec-4ccc-ae45-e0c20fd85593 None None] MFA Rules not processed for user `7e6f70cd24b7424ab0cf94c494ebdcbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:32.068724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c5964b34-70ec-4ccc-ae45-e0c20fd85593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:32.164677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1220/2439] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:32.176852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:32.183432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:32.254808 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada0018-8377-49d4-ae33-1a349de2982f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:32.259541 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95204bd338a4a99a510a67c4ab97532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:32.260425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:32.260706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:32.261886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:32.338108 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:32.339518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:32.350056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1220/2440] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:18:32.616435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:32.616726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:32.617009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:32.639530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:32.640789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:32.701816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1221/2441] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:18:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:32.836727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:32.838223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:32.884677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:32.908082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbd890-8179-4572-bfc3-91bc73463747 service glance] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:32.913849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:32.915103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:32.915500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:32.915812 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:32.923977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:32.925547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:32.934713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1221/2442] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:18:34.812524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:34.814006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:34.895229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:18:34.902871 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285768a-4cb4-4328-aa22-0cdef3ebc936 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:34.907610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b2305-cee0-4aba-a7cc-652ddeb60135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:34.909478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b2305-cee0-4aba-a7cc-652ddeb60135 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:34.909814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b2305-cee0-4aba-a7cc-652ddeb60135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:34.915550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b2305-cee0-4aba-a7cc-652ddeb60135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:34.915550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-016b2305-cee0-4aba-a7cc-652ddeb60135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:35.026949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b2305-cee0-4aba-a7cc-652ddeb60135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:35.028630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b2305-cee0-4aba-a7cc-652ddeb60135 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:35.060464 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1222/2443] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:18:37.853061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:37.878381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bd2bd-c82f-4c51-ad59-020ba6784150 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:37.884861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34142b-9b06-4518-82df-a1a53ca6c1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:37.885542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34142b-9b06-4518-82df-a1a53ca6c1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:37.885853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34142b-9b06-4518-82df-a1a53ca6c1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:37.886130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34142b-9b06-4518-82df-a1a53ca6c1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:37.887899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3f34142b-9b06-4518-82df-a1a53ca6c1a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:37.902452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34142b-9b06-4518-82df-a1a53ca6c1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:37.903893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34142b-9b06-4518-82df-a1a53ca6c1a0 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:37.915559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1222/2444] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:18:42.840922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b52607-4216-407a-9d45-2baee9ab71bb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:42.841558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b52607-4216-407a-9d45-2baee9ab71bb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:42.841558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b52607-4216-407a-9d45-2baee9ab71bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:42.864966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-03b52607-4216-407a-9d45-2baee9ab71bb None None] MFA Rules not processed for user `6e5c22b631df4600b9f920120b67fc67`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:42.868593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03b52607-4216-407a-9d45-2baee9ab71bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:42.937643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1223/2445] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:18:44.178869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:44.182726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:44.182726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:44.207024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] MFA Rules not processed for user `1623e93ad6844cacae7cc1a71b339f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:44.208490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:44.277411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1223/2446] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:46.581842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f7f4-a948-4e15-94db-792919cad851 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:46.582244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f7f4-a948-4e15-94db-792919cad851 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:46.582408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f7f4-a948-4e15-94db-792919cad851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:46.603673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-48a6f7f4-a948-4e15-94db-792919cad851 None None] MFA Rules not processed for user `9b17d0d51b034064a98bdae4bbc20fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:46.604951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-48a6f7f4-a948-4e15-94db-792919cad851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:46.612055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1224/2447] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:18:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:46.728473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:46.730040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:46.747280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f7159-fca2-4850-80ab-345300d5aa05 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:46.752830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:46.753432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:46.753750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:46.754040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:46.755749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:46.766907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:46.768838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:46.778723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1224/2448] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:18:51.150355 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0134b-ef25-4897-b8bc-d5bda4f6359c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:51.151019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0134b-ef25-4897-b8bc-d5bda4f6359c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:51.151019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0134b-ef25-4897-b8bc-d5bda4f6359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:51.173109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9ee0134b-ef25-4897-b8bc-d5bda4f6359c None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:18:51.174835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9ee0134b-ef25-4897-b8bc-d5bda4f6359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:51.193713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1225/2449] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:18:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:18:51.202680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:51.221258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a4f54-9511-4ed2-a27b-ad947c88e712 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:51.223073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:51.223680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:51.223959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:51.224235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:51.234338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:51.235914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:51.245626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1225/2450] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:18:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:18:54.361288 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0134b-ef25-4897-b8bc-d5bda4f6359c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:54.377919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0134b-ef25-4897-b8bc-d5bda4f6359c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:54.413737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:54.413737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:54.413737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:54.413737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:54.415530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:54.496437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:54.498399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:54.510413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1226/2451] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:18:54.822316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:18:54.833955 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d68da2-00f1-4a74-8f37-72c0e62bcad1 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:18:54.839481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:18:54.840077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:18:54.840355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:18:54.840817 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:18:54.842552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:18:54.854598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:18:54.856181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:18:54.865462 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1226/2452] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:18:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:19:01.066051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:19:01.069861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:19:01.077292 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae498d4-7a07-4983-bde3-37b6d9e1b30f service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:19:01.082380 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef47730-d6ba-44f8-a744-1f10fd857573 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:19:01.084752 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb21f3a-2b1f-40b2-a1a1-82dd62a8b361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:19:01.085423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb21f3a-2b1f-40b2-a1a1-82dd62a8b361 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:19:01.085787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb21f3a-2b1f-40b2-a1a1-82dd62a8b361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:19:01.086063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb21f3a-2b1f-40b2-a1a1-82dd62a8b361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:19:01.087354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:19:01.087939 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:19:01.088204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:19:01.088477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:19:01.089070 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9cb21f3a-2b1f-40b2-a1a1-82dd62a8b361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:19:01.091150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:19:01.102028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:19:01.104545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:19:01.104674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb21f3a-2b1f-40b2-a1a1-82dd62a8b361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:19:01.106349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb21f3a-2b1f-40b2-a1a1-82dd62a8b361 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:19:01.115788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1227/2453] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:19:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:19:01.117015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:19:01.119973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1227/2454] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:19:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:19:01.123218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:19:01.128612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374586d-0c7c-4a2f-a5c9-a59e7ebe78af service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:19:01.131132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:19:01.131809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:19:01.132207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:19:01.132482 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:19:01.141213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:19:01.142794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:19:01.153881 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1228/2455] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:19:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:19:03.799761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:19:03.800279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:19:03.800467 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:19:03.808924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=68 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:19:03.820966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] MFA Rules not processed for user `a9d55c52717546b09d93d956a29f49a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:19:03.823145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:19:03.830285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1228/2456] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:19:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:19:11.245029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:19:11.372960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:19:11.386586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6762a5-3e53-4bc5-991b-8dd001da4705 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:19:11.396077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:19:11.397175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:19:11.397645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:19:11.398063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:19:11.400499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:19:11.416821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:19:11.418483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:19:11.430441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1229/2457] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:19:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:19:13.827511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:19:14.822879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:19:14.824579 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:19:14.842709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c0716-c342-4490-8798-2fb5e49308ae None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:19:14.849040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2092ea1-5a6f-403a-a174-b30422112de1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:19:14.849919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092ea1-5a6f-403a-a174-b30422112de1 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:19:14.850355 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092ea1-5a6f-403a-a174-b30422112de1 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:19:14.850810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092ea1-5a6f-403a-a174-b30422112de1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:19:14.853095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a2092ea1-5a6f-403a-a174-b30422112de1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:19:14.866379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2092ea1-5a6f-403a-a174-b30422112de1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:19:14.868714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2092ea1-5a6f-403a-a174-b30422112de1 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:19:14.881749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1229/2458] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:19:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:19:21.425968 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:19:24.876604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:19:27.397219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:19:27.407332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887201-83d7-4c23-aed7-ab98d4d88420 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:19:27.411097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:19:27.411696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:19:27.411981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:19:27.412256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:19:27.421846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:19:27.423273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:19:27.432717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1230/2459] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:19:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:19:37.428602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:06.084569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:06.087712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:06.087712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:06.106983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] MFA Rules not processed for user `b694f1163ef54b82b86e06bf825c57e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:06.108281 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:06.165791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1230/2460] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:06.514979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:06.525403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e35064-0542-4c0d-b015-027cadc6f824 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:06.529600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:06.530325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:06.530837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:06.531212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:06.533271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:06.545081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:06.546756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:06.556105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1231/2461] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:07.620398 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:07.622469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:07.708562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307443e8-8a38-4788-8dd1-31d6bae3098c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:07.713834 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:07.714450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:07.714733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:07.715009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:07.718132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:07.729965 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:07.732767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:07.743103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1231/2462] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:16.552677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:16.933999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:16.953812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3781de-4cdf-4645-a332-fb5647c65ea2 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:16.961199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:16.961788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:16.962403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:16.962735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:16.964625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:16.990938 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:16.992668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:17.004049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1232/2463] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:17.738866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=57 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:18.766466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:18.778598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc0b33-9525-4d0d-9b7c-630e83816524 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:18.783362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b3ad2-0939-42f6-a9b2-c2c104fcdac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:18.783977 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b3ad2-0939-42f6-a9b2-c2c104fcdac8 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:18.784257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b3ad2-0939-42f6-a9b2-c2c104fcdac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:18.784531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b3ad2-0939-42f6-a9b2-c2c104fcdac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:18.786329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-413b3ad2-0939-42f6-a9b2-c2c104fcdac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:18.797527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b3ad2-0939-42f6-a9b2-c2c104fcdac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:18.802976 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b3ad2-0939-42f6-a9b2-c2c104fcdac8 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:18.813626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1232/2464] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:23.055697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:23.065301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb69c-5a13-4c11-9ba4-72d665a557fe service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:23.069125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:23.071804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:23.071804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:23.071804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:23.072399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:23.147362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:23.148853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:23.159111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1233/2465] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:24.220056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd787044-7e29-49cc-8fda-82f631ee5673 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:24.220626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd787044-7e29-49cc-8fda-82f631ee5673 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:24.220626 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd787044-7e29-49cc-8fda-82f631ee5673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:24.244417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bd787044-7e29-49cc-8fda-82f631ee5673 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:24.246267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bd787044-7e29-49cc-8fda-82f631ee5673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:24.258075 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1233/2466] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:20:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:24.345973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:24.359572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8a050-ffd6-4c17-8d8a-acefeddb7fca service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:24.365250 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:24.365913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:24.366204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:24.366490 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:24.375804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:24.377719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:24.397833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1234/2467] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:20:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:20:27.507873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:27.508520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:27.508520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:27.543848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] MFA Rules not processed for user `6e5c22b631df4600b9f920120b67fc67`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:27.557133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:27.570233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1234/2468] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:28.314842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:28.316976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:28.366362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48349ea9-7a11-47c5-af11-0415b67ef1b6 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:28.377278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:28.380677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:28.381138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:28.381812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:28.388969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:28.416751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:28.418569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:28.464683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1235/2469] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:28.552035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:28.553025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:28.573065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9b064-0337-4369-ba5c-054a0440f30f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:28.579844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:28.580559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:28.580776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:28.581590 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] PATH_INFO: `/v3/users/a9d55c52717546b09d93d956a29f49a9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:28.583481 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d55c52717546b09d93d956a29f49a9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:28.586357 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:28.904632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d55c52717546b09d93d956a29f49a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:28.908343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d55c52717546b09d93d956a29f49a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:28.989839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d55c52717546b09d93d956a29f49a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:28.994174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d55c52717546b09d93d956a29f49a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:29.012867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-55c90394-da34-4f94-9c6b-c7462dc474e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d55c52717546b09d93d956a29f49a9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:29.056613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1235/2470] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:20:28 2022] DELETE /identity/v3/users/a9d55c52717546b09d93d956a29f49a9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:29.547275 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:29.557954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bf16b-cc7b-403c-8ec7-fa2a6a7ebcc0 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:29.636166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:29.637098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:29.637098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:29.637342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] PATH_INFO: `/v3/projects/cb043a0ecad3466bbd5a5874126d2810` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:29.639443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb043a0ecad3466bbd5a5874126d2810)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:29.640971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:29.806286 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb043a0ecad3466bbd5a5874126d2810', 'request_id': 'req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:29.821688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:29.821688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb043a0ecad3466bbd5a5874126d2810', 'request_id': 'req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:29.836085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb043a0ecad3466bbd5a5874126d2810', 'request_id': 'req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:29.953497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] The token cache is being invalidate because project cb043a0ecad3466bbd5a5874126d2810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:20:29.957162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:29.957162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f72d1c4-3a7a-48d0-bcf2-63556e4000ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:29.962074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd57e07-2d9a-4234-bebc-2019586b4fc2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:29.962625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd57e07-2d9a-4234-bebc-2019586b4fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:29.962928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd57e07-2d9a-4234-bebc-2019586b4fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:29.965791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1236/2471] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:20:29 2022] DELETE /identity/v3/projects/cb043a0ecad3466bbd5a5874126d2810 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:29.987043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dcd57e07-2d9a-4234-bebc-2019586b4fc2 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:29.988314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dcd57e07-2d9a-4234-bebc-2019586b4fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:30.103181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1236/2472] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:30.119922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e4c9a-32e7-4638-9923-0e34f6a4016b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.120289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e4c9a-32e7-4638-9923-0e34f6a4016b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.120647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e4c9a-32e7-4638-9923-0e34f6a4016b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:30.142925 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-101e4c9a-32e7-4638-9923-0e34f6a4016b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:30.144460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-101e4c9a-32e7-4638-9923-0e34f6a4016b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:30.153305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1237/2473] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:30.158750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd57e07-2d9a-4234-bebc-2019586b4fc2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:30.169723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd57e07-2d9a-4234-bebc-2019586b4fc2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:30.174586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:30.175197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.175495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.175783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:30.176469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:20:30.176774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:30.178534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:30.203362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1237/2474] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:20:30.206305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e4c9a-32e7-4638-9923-0e34f6a4016b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:30.219404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e4c9a-32e7-4638-9923-0e34f6a4016b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:30.240200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:30.240995 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.241279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.241575 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:30.242244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:30.244209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:30.379282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1238/2475] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:20:30 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:30.383365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:30.400319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ab7cb-56d0-4b63-b5dc-6acddc2b56f3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:30.450243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:30.485227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:30.486195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.486678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.487145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:30.488027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:30.490339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:30.592592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1238/2476] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:20:30 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:30.597055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:30.614736 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8277c8-132c-4743-aeeb-8fe6341201a0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:30.629847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:30.630646 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.631804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.632247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:30.633062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:30.635040 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:30.660921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1239/2477] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:30.665076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:30.690901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ab9e-f052-49ef-ab70-aa973a8b4246 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:30.704307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ba794-c439-4e36-93f0-1573b13ac7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:30.706782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba794-c439-4e36-93f0-1573b13ac7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.706782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba794-c439-4e36-93f0-1573b13ac7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.706782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba794-c439-4e36-93f0-1573b13ac7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:30.706782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ba794-c439-4e36-93f0-1573b13ac7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:30.712777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ba794-c439-4e36-93f0-1573b13ac7e6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:30.739520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1239/2478] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:30.739738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:30.752342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291da3f-7789-451f-915d-23b09d9690f3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:30.760258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:30.761539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.762098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.762610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] PATH_INFO: `/v3/projects/d35c1232882d47c680da29578c0ab99c/users/c1e4625561e0499686e1d9c661bdbbcd/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:30.829023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35c1232882d47c680da29578c0ab99c, user_id=c1e4625561e0499686e1d9c661bdbbcd, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:30.843845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:30.949773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1240/2479] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:30 2022] PUT /identity/v3/projects/d35c1232882d47c680da29578c0ab99c/users/c1e4625561e0499686e1d9c661bdbbcd/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:30.961341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9d24-d89d-4ac2-abaa-19e4262a9b0c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:30.961783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9d24-d89d-4ac2-abaa-19e4262a9b0c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:30.962068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9d24-d89d-4ac2-abaa-19e4262a9b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:31.053277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-721c9d24-d89d-4ac2-abaa-19e4262a9b0c None None] MFA Rules not processed for user `c1e4625561e0499686e1d9c661bdbbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:31.060654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-721c9d24-d89d-4ac2-abaa-19e4262a9b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:31.230712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1240/2480] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:31.236876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:31.367788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58e862-238d-4343-96d0-95a21f903b9b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:31.384142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:31.384759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:31.387564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:31.387564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:31.387564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:31.390507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:31.475087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1241/2481] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:20:31 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:31.479337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9d24-d89d-4ac2-abaa-19e4262a9b0c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:31.492317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9d24-d89d-4ac2-abaa-19e4262a9b0c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:31.562804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:31.563613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:31.563960 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:31.564332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:31.565085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:31.567249 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:31.693813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1241/2482] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:20:31 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:31.695918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:31.710135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342033c4-fd45-4b22-b8c0-c47c59981774 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:31.716304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:31.724748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:31.728942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:31.729445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:31.730385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:31.732286 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:31.753485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1242/2483] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:31.758115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:31.770900 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1185e20-acd2-4431-a053-71b9ace3e050 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:31.791686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:31.792695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:31.793080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:31.793428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] PATH_INFO: `/v3/projects/795b3cb11cae4372ad284443f2b6b6c8/users/f4cdd45195e840c493087719a7266c64/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:31.815340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=795b3cb11cae4372ad284443f2b6b6c8, user_id=f4cdd45195e840c493087719a7266c64, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:31.818381 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:31.922885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1242/2484] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:31 2022] PUT /identity/v3/projects/795b3cb11cae4372ad284443f2b6b6c8/users/f4cdd45195e840c493087719a7266c64/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:31.926926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:31.941053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643dae0-6b35-4ce9-afd5-ef2f520e533e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:31.997132 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:32.006780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa0b73-9e63-47a6-888c-6d312869572e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:32.010192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa0b73-9e63-47a6-888c-6d312869572e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:32.010192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa0b73-9e63-47a6-888c-6d312869572e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:32.010192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa0b73-9e63-47a6-888c-6d312869572e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:32.010192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa0b73-9e63-47a6-888c-6d312869572e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:32.011802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa0b73-9e63-47a6-888c-6d312869572e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:32.026239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1243/2485] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:32.032163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:32.044447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4260f-6927-4566-b344-f0fb6e9118dd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:32.051958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:32.052634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:32.052917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:32.053199 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] PATH_INFO: `/v3/projects/795b3cb11cae4372ad284443f2b6b6c8/users/f4cdd45195e840c493087719a7266c64/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:32.055536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795b3cb11cae4372ad284443f2b6b6c8, user_id=f4cdd45195e840c493087719a7266c64, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:32.058232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:32.060979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:32.070912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1243/2486] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:32 2022] PUT /identity/v3/projects/795b3cb11cae4372ad284443f2b6b6c8/users/f4cdd45195e840c493087719a7266c64/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:32.088317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bad06-ebf8-471b-87e8-4cd4496054c2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:32.088859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bad06-ebf8-471b-87e8-4cd4496054c2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:32.089318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bad06-ebf8-471b-87e8-4cd4496054c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:32.165456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-401bad06-ebf8-471b-87e8-4cd4496054c2 None None] MFA Rules not processed for user `f4cdd45195e840c493087719a7266c64`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:32.166751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-401bad06-ebf8-471b-87e8-4cd4496054c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:32.333601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1244/2487] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:32.345839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:32.347958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:32.455914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce2b2-0fba-4204-96d4-73d628276ed8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:32.461072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87792792-7583-406f-bf3e-22fe3c6a4777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:32.463011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87792792-7583-406f-bf3e-22fe3c6a4777 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:32.463400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87792792-7583-406f-bf3e-22fe3c6a4777 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:32.467760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87792792-7583-406f-bf3e-22fe3c6a4777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:32.477131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87792792-7583-406f-bf3e-22fe3c6a4777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:32.479150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87792792-7583-406f-bf3e-22fe3c6a4777 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:32.493543 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1244/2488] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:32.528114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bad06-ebf8-471b-87e8-4cd4496054c2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:32.667510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bad06-ebf8-471b-87e8-4cd4496054c2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:32.679255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:32.680393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:32.680787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:32.681161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] PATH_INFO: `/v3/users/c1e4625561e0499686e1d9c661bdbbcd` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:32.683833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e4625561e0499686e1d9c661bdbbcd)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:32.686504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:32.978411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e4625561e0499686e1d9c661bdbbcd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:32.978411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e4625561e0499686e1d9c661bdbbcd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:33.001979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e4625561e0499686e1d9c661bdbbcd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:33.159313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e4625561e0499686e1d9c661bdbbcd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:33.160128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e4625561e0499686e1d9c661bdbbcd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:33.168554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:33.208271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1245/2489] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:20:32 2022] DELETE /identity/v3/users/c1e4625561e0499686e1d9c661bdbbcd => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:33.213450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:33.213450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:33.214135 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:33.253704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:33.253704 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:33.354045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1245/2490] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:33.364859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:33.366354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:33.430695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28a0e-3c44-48e8-b498-1f3e2c718221 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:33.434966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:33.435583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:33.436484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:33.436919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:33.450541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:33.452924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:33.462853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1246/2491] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:34.293239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:34.297295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:34.317255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505977e-3e84-41f4-bb86-c0d00bfd7a01 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:34.335527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:34.338958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:34.338958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:34.338958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] PATH_INFO: `/v3/projects/d35c1232882d47c680da29578c0ab99c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:34.338958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d35c1232882d47c680da29578c0ab99c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:34.339919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:34.505345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd35c1232882d47c680da29578c0ab99c', 'request_id': 'req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:34.594177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd35c1232882d47c680da29578c0ab99c', 'request_id': 'req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:34.598413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd35c1232882d47c680da29578c0ab99c', 'request_id': 'req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:34.622935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] The token cache is being invalidate because project d35c1232882d47c680da29578c0ab99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:20:34.624225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:34.625431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:34.627531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1246/2492] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:20:34 2022] DELETE /identity/v3/projects/d35c1232882d47c680da29578c0ab99c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:34.631477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:34.646370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d2660-bb2d-483b-91d3-e0d87917d3bc service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:34.746287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:34.747321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:34.747810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:34.748086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] PATH_INFO: `/v3/users/f4cdd45195e840c493087719a7266c64` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:34.749966 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4cdd45195e840c493087719a7266c64)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:34.752005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:34.959658 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4cdd45195e840c493087719a7266c64'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:34.960416 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4cdd45195e840c493087719a7266c64'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:34.975267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4cdd45195e840c493087719a7266c64'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:35.051007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4cdd45195e840c493087719a7266c64'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:35.051812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-94452cca-6beb-4074-ab1b-c30688473496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4cdd45195e840c493087719a7266c64'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:35.072185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1247/2493] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:20:34 2022] DELETE /identity/v3/users/f4cdd45195e840c493087719a7266c64 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:35.598620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:35.598620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:35.631614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4f9ef-73bc-4c8c-869d-9de72bdb80ca admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:35.656182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:35.740361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:35.741465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:35.741946 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:35.743271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] PATH_INFO: `/v3/projects/795b3cb11cae4372ad284443f2b6b6c8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:35.745239 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=795b3cb11cae4372ad284443f2b6b6c8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:35.747696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:35.929464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795b3cb11cae4372ad284443f2b6b6c8', 'request_id': 'req-73944959-36a2-41a5-8364-66c257dd2666'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:36.018341 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795b3cb11cae4372ad284443f2b6b6c8', 'request_id': 'req-73944959-36a2-41a5-8364-66c257dd2666'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:36.026024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795b3cb11cae4372ad284443f2b6b6c8', 'request_id': 'req-73944959-36a2-41a5-8364-66c257dd2666'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:36.050143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] The token cache is being invalidate because project 795b3cb11cae4372ad284443f2b6b6c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:20:36.050684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:36.051340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73944959-36a2-41a5-8364-66c257dd2666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:36.056988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1247/2494] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:20:35 2022] DELETE /identity/v3/projects/795b3cb11cae4372ad284443f2b6b6c8 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:36.064000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392baa-defb-4929-b82a-5e87fe1836a9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.064000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392baa-defb-4929-b82a-5e87fe1836a9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.064000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74392baa-defb-4929-b82a-5e87fe1836a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.079676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-74392baa-defb-4929-b82a-5e87fe1836a9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:36.081328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74392baa-defb-4929-b82a-5e87fe1836a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:36.098706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:36.205638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1248/2495] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:36.211193 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c70de-6276-414f-9a89-f2961dae16b6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.211714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c70de-6276-414f-9a89-f2961dae16b6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.211999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c70de-6276-414f-9a89-f2961dae16b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.248241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-da9c70de-6276-414f-9a89-f2961dae16b6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:36.249835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-da9c70de-6276-414f-9a89-f2961dae16b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:36.263956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1248/2496] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:36.268431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74392baa-defb-4929-b82a-5e87fe1836a9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:36.279274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74392baa-defb-4929-b82a-5e87fe1836a9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:36.285382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:36.286081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.286383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.286831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.287519 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:20:36.287835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:36.289712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:36.304056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1249/2497] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:20:36.311453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c70de-6276-414f-9a89-f2961dae16b6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:36.338033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c70de-6276-414f-9a89-f2961dae16b6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:36.344937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:36.345816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.347956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.349594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.350519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:36.352754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:36.449202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1249/2498] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:20:36 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:36.450637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:36.464303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111ace2-f6a3-45c7-9ac6-a5cb8b8dfd1f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:36.548397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:36.549028 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.549352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.549678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.550326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:36.552854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:36.696301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1250/2499] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:20:36 2022] POST /identity/v3/users => generated 292 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:36.697443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:36.711452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47575c-d1dd-40d7-afac-33e4d50c4188 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:36.729457 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:36.730546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.730831 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.731103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.731875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:36.734935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:36.751006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1250/2500] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:20:36.753958 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:36.769476 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7f553-7290-49d6-95ce-e415e2044f97 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:36.775948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02b5d2-01fd-465b-922c-8445cd4f0ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:36.776768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02b5d2-01fd-465b-922c-8445cd4f0ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.777145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02b5d2-01fd-465b-922c-8445cd4f0ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.777681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02b5d2-01fd-465b-922c-8445cd4f0ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.778451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02b5d2-01fd-465b-922c-8445cd4f0ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:36.781062 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02b5d2-01fd-465b-922c-8445cd4f0ad5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:36.793315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1251/2501] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:20:36.797230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:36.809602 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee4998-a995-403d-977a-8cc8282e02f4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:36.815448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:36.816200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.816568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.817085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] PATH_INFO: `/v3/projects/036c11564f3141fb98a59369533a10b2/users/69406ad6e1cf44978fb9e7a166a16a87/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:36.840027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036c11564f3141fb98a59369533a10b2, user_id=69406ad6e1cf44978fb9e7a166a16a87, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:36.841688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:36.966315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1251/2502] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:36 2022] PUT /identity/v3/projects/036c11564f3141fb98a59369533a10b2/users/69406ad6e1cf44978fb9e7a166a16a87/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:36.977122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8f0e3-6870-4e8a-989c-3a2190e2cf3a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:36.977300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8f0e3-6870-4e8a-989c-3a2190e2cf3a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:36.977761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8f0e3-6870-4e8a-989c-3a2190e2cf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:37.030285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b2d8f0e3-6870-4e8a-989c-3a2190e2cf3a None None] MFA Rules not processed for user `69406ad6e1cf44978fb9e7a166a16a87`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:37.032631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b2d8f0e3-6870-4e8a-989c-3a2190e2cf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:37.125972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1252/2503] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:37.130396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:37.200615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:37.219319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf258296-10d6-4d33-8452-edafe5dc42c9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:37.225217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:37.226426 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:37.226822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:37.227192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:37.232298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:37.235422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:37.340130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1252/2504] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:20:37 2022] POST /identity/v3/projects => generated 383 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:37.345142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8f0e3-6870-4e8a-989c-3a2190e2cf3a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:37.360074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8f0e3-6870-4e8a-989c-3a2190e2cf3a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:37.381271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:37.460379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:37.461029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:37.461315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:37.461623 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:37.462352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:37.464337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:37.934159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1253/2505] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:20:37 2022] POST /identity/v3/users => generated 298 bytes in 590 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:37.939159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:37.952877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3ace0-cd1c-404f-a24e-c0a89f800b75 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:37.955903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:37.956527 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:37.956803 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:37.957074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:37.957767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:37.959363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:37.974396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1253/2506] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:20:37.975556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:37.987342 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5537505-597c-4e41-8215-f3026b0a2afa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:37.992192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:37.992944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:37.993314 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:37.993690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] PATH_INFO: `/v3/projects/c4f07cf0174e46e6ab5640188a501de8/users/44afc2c7f4b34705971be02cb8bd5d1d/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:38.011361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f07cf0174e46e6ab5640188a501de8, user_id=44afc2c7f4b34705971be02cb8bd5d1d, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:38.015174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:38.094688 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1254/2507] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:37 2022] PUT /identity/v3/projects/c4f07cf0174e46e6ab5640188a501de8/users/44afc2c7f4b34705971be02cb8bd5d1d/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:38.099894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:38.113451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98527136-e76e-46dd-9d51-026c050e1859 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:38.202504 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998dbfd-10ac-44f0-b466-d547f36a0109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:38.203905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998dbfd-10ac-44f0-b466-d547f36a0109 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:38.204042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998dbfd-10ac-44f0-b466-d547f36a0109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:38.204374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998dbfd-10ac-44f0-b466-d547f36a0109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:38.205105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998dbfd-10ac-44f0-b466-d547f36a0109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:38.210997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998dbfd-10ac-44f0-b466-d547f36a0109 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:38.224232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1254/2508] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:38.234984 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:38.247625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e4fa9-97c9-4dee-a945-80faee1272a9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:38.253435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:38.254262 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:38.254672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:38.255088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] PATH_INFO: `/v3/projects/c4f07cf0174e46e6ab5640188a501de8/users/44afc2c7f4b34705971be02cb8bd5d1d/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:38.257331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f07cf0174e46e6ab5640188a501de8, user_id=44afc2c7f4b34705971be02cb8bd5d1d, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:38.260237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:38.278843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1255/2509] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:38 2022] PUT /identity/v3/projects/c4f07cf0174e46e6ab5640188a501de8/users/44afc2c7f4b34705971be02cb8bd5d1d/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:20:38.290723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee904a-3bd0-405c-8176-82df982d99c3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:38.291124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee904a-3bd0-405c-8176-82df982d99c3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:38.291503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee904a-3bd0-405c-8176-82df982d99c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:38.334309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5cee904a-3bd0-405c-8176-82df982d99c3 None None] MFA Rules not processed for user `44afc2c7f4b34705971be02cb8bd5d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:38.335500 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5cee904a-3bd0-405c-8176-82df982d99c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:38.424952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1255/2510] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:38.439699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:38.442468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:38.527121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede077ca-0728-411c-b662-038038a0ecf1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:38.533272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b845cbb-0863-4dd7-a13c-5148b4d5cfcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:38.534518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b845cbb-0863-4dd7-a13c-5148b4d5cfcf service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:38.535582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b845cbb-0863-4dd7-a13c-5148b4d5cfcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:38.535962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b845cbb-0863-4dd7-a13c-5148b4d5cfcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:38.608767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b845cbb-0863-4dd7-a13c-5148b4d5cfcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:38.610287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b845cbb-0863-4dd7-a13c-5148b4d5cfcf service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:38.625790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1256/2511] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:44.300720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee904a-3bd0-405c-8176-82df982d99c3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:44.310059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee904a-3bd0-405c-8176-82df982d99c3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:44.314932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:44.318294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:44.318685 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:44.318890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:44.330854 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:44.332464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:44.343149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1256/2512] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:48.620826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:50.095638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d85d57-f8c4-47bc-a7e6-04171b6ac27b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:50.096251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d85d57-f8c4-47bc-a7e6-04171b6ac27b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:50.096251 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d85d57-f8c4-47bc-a7e6-04171b6ac27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:50.115077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d3d85d57-f8c4-47bc-a7e6-04171b6ac27b None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:50.116260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d3d85d57-f8c4-47bc-a7e6-04171b6ac27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:50.169993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1257/2513] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:20:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:50.181782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:50.183231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:50.216872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:50.253894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9df8b-c28e-4c1e-8106-af7d6a2576fa service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:50.259436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a55c3b-faa9-4695-82ac-1a28870c08b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:50.260038 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a55c3b-faa9-4695-82ac-1a28870c08b1 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:50.260347 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a55c3b-faa9-4695-82ac-1a28870c08b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:50.260621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a55c3b-faa9-4695-82ac-1a28870c08b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:50.269515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a55c3b-faa9-4695-82ac-1a28870c08b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:50.273200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a55c3b-faa9-4695-82ac-1a28870c08b1 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:50.279277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1257/2514] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:20:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:20:50.458448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d85d57-f8c4-47bc-a7e6-04171b6ac27b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:50.554362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d85d57-f8c4-47bc-a7e6-04171b6ac27b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:50.560614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:50.561786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:50.562085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:50.562447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] PATH_INFO: `/v3/users/69406ad6e1cf44978fb9e7a166a16a87` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:50.566371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69406ad6e1cf44978fb9e7a166a16a87)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:50.570110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:50.834295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69406ad6e1cf44978fb9e7a166a16a87'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:50.835008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69406ad6e1cf44978fb9e7a166a16a87'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:50.849797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69406ad6e1cf44978fb9e7a166a16a87'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:50.904199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69406ad6e1cf44978fb9e7a166a16a87'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:50.905038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69406ad6e1cf44978fb9e7a166a16a87'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:50.919819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1258/2515] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:20:50 2022] DELETE /identity/v3/users/69406ad6e1cf44978fb9e7a166a16a87 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:50.925303 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:50.925740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:50.926116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:50.955556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:50.956889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:51.026751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1258/2516] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:51.037682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:51.039189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:51.075111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:51.107359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5166-c41e-45aa-bcd9-ad3fa94de480 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:51.111929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:51.112552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:51.112835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:51.113110 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:51.123086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:51.124514 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:51.136362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1259/2517] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:51.581229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:51.583810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:51.596703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12b40b-81b2-4495-806f-4499f2d2e4d3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:51.603074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:51.603967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:51.605051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:51.605724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] PATH_INFO: `/v3/projects/036c11564f3141fb98a59369533a10b2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:51.607150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=036c11564f3141fb98a59369533a10b2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:51.609368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:51.757737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036c11564f3141fb98a59369533a10b2', 'request_id': 'req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:51.827790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036c11564f3141fb98a59369533a10b2', 'request_id': 'req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:51.834868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036c11564f3141fb98a59369533a10b2', 'request_id': 'req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:51.847982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] The token cache is being invalidate because project 036c11564f3141fb98a59369533a10b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:20:51.848277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:51.848951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:51.850479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1259/2518] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:20:51 2022] DELETE /identity/v3/projects/036c11564f3141fb98a59369533a10b2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:51.854555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:51.864510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0dde5-4821-4f96-b243-9e3c83e9155e service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:51.934598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:51.935163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:51.935448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:51.935727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] PATH_INFO: `/v3/users/44afc2c7f4b34705971be02cb8bd5d1d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:51.937000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44afc2c7f4b34705971be02cb8bd5d1d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:51.939370 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:52.139174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44afc2c7f4b34705971be02cb8bd5d1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:52.139847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44afc2c7f4b34705971be02cb8bd5d1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:52.153056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44afc2c7f4b34705971be02cb8bd5d1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:52.223261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44afc2c7f4b34705971be02cb8bd5d1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:52.223261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1931b14b-246c-440a-a2c7-ece4709ff525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44afc2c7f4b34705971be02cb8bd5d1d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:52.232807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1260/2519] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:20:51 2022] DELETE /identity/v3/users/44afc2c7f4b34705971be02cb8bd5d1d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:53.128814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:53.128814 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:53.144136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208501-0a6f-4ff3-8fb8-760e0de8d9eb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:53.192677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:53.210904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:53.211471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.211747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.212016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] PATH_INFO: `/v3/projects/c4f07cf0174e46e6ab5640188a501de8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.213230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f07cf0174e46e6ab5640188a501de8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:53.215258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:53.327043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f07cf0174e46e6ab5640188a501de8', 'request_id': 'req-44d241b5-d0be-4804-8472-68f3e687e8fe'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:53.415989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f07cf0174e46e6ab5640188a501de8', 'request_id': 'req-44d241b5-d0be-4804-8472-68f3e687e8fe'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:53.422667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f07cf0174e46e6ab5640188a501de8', 'request_id': 'req-44d241b5-d0be-4804-8472-68f3e687e8fe'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:53.440610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] The token cache is being invalidate because project c4f07cf0174e46e6ab5640188a501de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:20:53.441506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:53.442191 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44d241b5-d0be-4804-8472-68f3e687e8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:53.444155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1260/2520] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:20:53 2022] DELETE /identity/v3/projects/c4f07cf0174e46e6ab5640188a501de8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:53.449692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088fb5-a011-4a2e-a49d-c8f838558d5a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.450151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088fb5-a011-4a2e-a49d-c8f838558d5a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.450425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088fb5-a011-4a2e-a49d-c8f838558d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.474308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-30088fb5-a011-4a2e-a49d-c8f838558d5a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:53.476136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30088fb5-a011-4a2e-a49d-c8f838558d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:53.526768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:53.552399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1261/2521] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:53.556597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356dac-750d-4e4b-9c49-71acd5e2ae11 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.557042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356dac-750d-4e4b-9c49-71acd5e2ae11 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.557388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356dac-750d-4e4b-9c49-71acd5e2ae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.579797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d0356dac-750d-4e4b-9c49-71acd5e2ae11 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:53.585418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d0356dac-750d-4e4b-9c49-71acd5e2ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:53.591313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1261/2522] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:20:53.596777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30088fb5-a011-4a2e-a49d-c8f838558d5a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:53.607809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30088fb5-a011-4a2e-a49d-c8f838558d5a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:53.613061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:53.613790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.614131 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.614400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.615159 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:20:53.615472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:53.618000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:53.634106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1262/2523] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:20:53.640761 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356dac-750d-4e4b-9c49-71acd5e2ae11 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:53.652064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356dac-750d-4e4b-9c49-71acd5e2ae11 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:53.657037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:53.657703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.657985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.658251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.659539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:53.661502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:53.735471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1262/2524] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:20:53 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:53.740588 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:53.753633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c9592-4822-424d-862f-8f66f6624b9e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:53.812468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:53.813076 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.813358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.813667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.814319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:53.816902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:53.899336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1263/2525] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:20:53 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:20:53.904436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:53.917103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7460dd3-274b-4a77-94df-c2c6aecd80d6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:53.922115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:53.922723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.923002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.923276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.923928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:53.926136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:53.946219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1263/2526] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:53.950199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:53.961272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c7f6b-ead5-4a55-90d9-27971514d581 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:53.966654 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5b4a8-a38a-43c3-a2c9-f470ceb53935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:53.967330 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5b4a8-a38a-43c3-a2c9-f470ceb53935 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:53.967657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5b4a8-a38a-43c3-a2c9-f470ceb53935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:53.967967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5b4a8-a38a-43c3-a2c9-f470ceb53935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:53.968626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5b4a8-a38a-43c3-a2c9-f470ceb53935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:53.970458 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5b4a8-a38a-43c3-a2c9-f470ceb53935 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:53.983329 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1264/2527] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:20:53.988321 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:53.998878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee9c09-4a97-4c83-b49d-556723a19d74 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:54.004689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:54.005358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:54.005671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:54.005956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] PATH_INFO: `/v3/projects/7e371b50eae549798a892b9f4f71de61/users/f77e9709bd764880bb8a57fb441bc976/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:54.022043 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e371b50eae549798a892b9f4f71de61, user_id=f77e9709bd764880bb8a57fb441bc976, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:54.031449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:54.104110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1264/2528] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:53 2022] PUT /identity/v3/projects/7e371b50eae549798a892b9f4f71de61/users/f77e9709bd764880bb8a57fb441bc976/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:54.117453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1e9fe-9f9a-4411-a65a-447711011111 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:54.117898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1e9fe-9f9a-4411-a65a-447711011111 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:54.118315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1e9fe-9f9a-4411-a65a-447711011111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:54.153174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b3a1e9fe-9f9a-4411-a65a-447711011111 None None] MFA Rules not processed for user `f77e9709bd764880bb8a57fb441bc976`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:54.154593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3a1e9fe-9f9a-4411-a65a-447711011111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:54.228382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1265/2529] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:20:54.240905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:54.242601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:54.307454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1184cb2-074b-45c6-84de-ab1c4784fb9c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:54.311197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c540c2-bcff-486b-8908-0a5d92fc8d92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:54.311833 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c540c2-bcff-486b-8908-0a5d92fc8d92 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:54.312117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c540c2-bcff-486b-8908-0a5d92fc8d92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:54.312393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c540c2-bcff-486b-8908-0a5d92fc8d92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:54.321876 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c540c2-bcff-486b-8908-0a5d92fc8d92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:54.324062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c540c2-bcff-486b-8908-0a5d92fc8d92 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:54.334027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1265/2530] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:54.356773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1e9fe-9f9a-4411-a65a-447711011111 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:54.402964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:54.421430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1e9fe-9f9a-4411-a65a-447711011111 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:54.426711 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:54.427352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:54.427638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:54.427919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] PATH_INFO: `/v3/users/f77e9709bd764880bb8a57fb441bc976` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:54.429080 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f77e9709bd764880bb8a57fb441bc976)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:54.431189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:54.643888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f77e9709bd764880bb8a57fb441bc976'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:54.644618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f77e9709bd764880bb8a57fb441bc976'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:54.658390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f77e9709bd764880bb8a57fb441bc976'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:54.750126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f77e9709bd764880bb8a57fb441bc976'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:54.750890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f77e9709bd764880bb8a57fb441bc976'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:54.766521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1266/2531] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:20:54 2022] DELETE /identity/v3/users/f77e9709bd764880bb8a57fb441bc976 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:54.771147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:54.771435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:54.771705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:54.791613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:54.792806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:54.797684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:54.855515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1266/2532] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:54.865661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:54.867201 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:54.931664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c38d-175e-4f54-8d2d-d16fef974d75 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:54.935248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8235892-8537-4cc3-b030-8b8554607318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:54.935861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8235892-8537-4cc3-b030-8b8554607318 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:54.936145 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8235892-8537-4cc3-b030-8b8554607318 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:54.936420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8235892-8537-4cc3-b030-8b8554607318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:54.946908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8235892-8537-4cc3-b030-8b8554607318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:54.948505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8235892-8537-4cc3-b030-8b8554607318 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:54.959491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1267/2533] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:20:55.472926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:55.474534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:55.488528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7374-a7ce-40cd-a92c-1d8901ae49f8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:55.494362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:55.495166 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:55.495451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:55.496177 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] PATH_INFO: `/v3/projects/7e371b50eae549798a892b9f4f71de61` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:55.497082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e371b50eae549798a892b9f4f71de61)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:55.499103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:55.610656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e371b50eae549798a892b9f4f71de61', 'request_id': 'req-fec014c2-6d96-4cf8-9fa7-51453b6e940c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:55.661767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e371b50eae549798a892b9f4f71de61', 'request_id': 'req-fec014c2-6d96-4cf8-9fa7-51453b6e940c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:55.668528 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e371b50eae549798a892b9f4f71de61', 'request_id': 'req-fec014c2-6d96-4cf8-9fa7-51453b6e940c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:55.681667 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] The token cache is being invalidate because project 7e371b50eae549798a892b9f4f71de61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:20:55.681966 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:55.682701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-fec014c2-6d96-4cf8-9fa7-51453b6e940c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:20:55.684259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1267/2534] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:20:55 2022] DELETE /identity/v3/projects/7e371b50eae549798a892b9f4f71de61 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:55.689554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc484-b372-4838-9131-86d22a83c53e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:55.691125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc484-b372-4838-9131-86d22a83c53e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:55.691125 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc484-b372-4838-9131-86d22a83c53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:55.711847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-749dc484-b372-4838-9131-86d22a83c53e None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:55.713073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-749dc484-b372-4838-9131-86d22a83c53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:55.779630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1268/2535] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:55.789052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a00d9-cc6d-4517-a1d9-6571b4b70cce None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:55.789420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a00d9-cc6d-4517-a1d9-6571b4b70cce None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:55.789797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a00d9-cc6d-4517-a1d9-6571b4b70cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:55.826072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-406a00d9-cc6d-4517-a1d9-6571b4b70cce None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:55.827279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-406a00d9-cc6d-4517-a1d9-6571b4b70cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:55.835566 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1268/2536] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:20:55.840006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc484-b372-4838-9131-86d22a83c53e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:55.847037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:55.851607 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc484-b372-4838-9131-86d22a83c53e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:55.856621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:55.858189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:55.858189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:55.858189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:55.858581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:20:55.858923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:55.860520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:55.874436 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1269/2537] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:20:55.880046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a00d9-cc6d-4517-a1d9-6571b4b70cce None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:55.892993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a00d9-cc6d-4517-a1d9-6571b4b70cce None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:55.898718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:55.899356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:55.899634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:55.899899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:55.900539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:55.902475 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:55.975905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1269/2538] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:20:55 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:55.980983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:55.992261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d204-a56d-4c8e-8b38-da832c11c929 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:56.057872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:56.058642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:56.059026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:56.059471 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:56.060516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:56.063077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:56.174294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1270/2539] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:20:55 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:56.179422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:56.189640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e13ae3-0245-4645-8ec9-e3840e578ec6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:56.194375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:56.194877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:56.195157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:56.195430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:56.196072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:56.197695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:56.212595 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1270/2540] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:56.217468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:56.229269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909ec8b-8330-4e9e-9f97-cae8e08d736c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:56.240845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7968d8-0b96-4604-9312-a0a71e0a9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:56.241913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7968d8-0b96-4604-9312-a0a71e0a9676 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:56.242530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7968d8-0b96-4604-9312-a0a71e0a9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:56.243044 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7968d8-0b96-4604-9312-a0a71e0a9676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:56.244174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7968d8-0b96-4604-9312-a0a71e0a9676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:56.247325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7968d8-0b96-4604-9312-a0a71e0a9676 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:56.261601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1271/2541] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:20:56.268547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:56.279245 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a6ab-f5aa-4738-a10b-751ab8a7e5a1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:56.284423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:56.285289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:56.285630 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:56.285908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] PATH_INFO: `/v3/projects/305c3e7ed45f41a382c0e3344d34b361/users/56e206e4ad424e8e965635d82a50eb5f/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:56.303727 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305c3e7ed45f41a382c0e3344d34b361, user_id=56e206e4ad424e8e965635d82a50eb5f, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:56.306479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:56.386972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1271/2542] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:56 2022] PUT /identity/v3/projects/305c3e7ed45f41a382c0e3344d34b361/users/56e206e4ad424e8e965635d82a50eb5f/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:56.395837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33992350-9367-4de6-bee2-5dca67a2af6e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:56.396149 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33992350-9367-4de6-bee2-5dca67a2af6e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:56.396426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33992350-9367-4de6-bee2-5dca67a2af6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:56.434710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-33992350-9367-4de6-bee2-5dca67a2af6e None None] MFA Rules not processed for user `56e206e4ad424e8e965635d82a50eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:56.436274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-33992350-9367-4de6-bee2-5dca67a2af6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:56.512346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1272/2543] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:56.516992 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:56.557875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:56.594176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebc982-6379-4ac6-9fd0-b7049bd564d3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:56.602017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:56.604169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:56.604169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:56.604169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:56.604622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:56.606300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:56.684868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1272/2544] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:20:56 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:20:56.688730 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33992350-9367-4de6-bee2-5dca67a2af6e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:56.700522 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33992350-9367-4de6-bee2-5dca67a2af6e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:56.772408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:56.778378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:56.778378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:56.778378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:56.778378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:56.778378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:56.791221 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:20:57.441990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1273/2545] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:20:56 2022] POST /identity/v3/users => generated 293 bytes in 754 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:20:57.446397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:57.458970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de847-b1fb-4b8e-8ce0-c48ef181a2e0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:57.463872 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:57.464477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:57.464757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:57.465120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:57.465921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:57.467691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:57.482851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1273/2546] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:20:57.487850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:57.502179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797dcce-e00f-467f-a2d0-c1c0afbe30fd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:57.508066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:57.508865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:57.509229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:57.509598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] PATH_INFO: `/v3/projects/b28579c61798424988aa02fe3a023294/users/6180f37cb0144f9f902305492aaa6f25/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:57.529772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28579c61798424988aa02fe3a023294, user_id=6180f37cb0144f9f902305492aaa6f25, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:57.532707 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:57.664835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1274/2547] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:57 2022] PUT /identity/v3/projects/b28579c61798424988aa02fe3a023294/users/6180f37cb0144f9f902305492aaa6f25/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:20:57.667986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:57.680329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84d07d-a9f2-46dc-a22d-9f3b55ad7c13 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:57.765616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429ac89-ff16-447d-b5a9-ccf568ee6604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:57.767780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429ac89-ff16-447d-b5a9-ccf568ee6604 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:57.768956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429ac89-ff16-447d-b5a9-ccf568ee6604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:57.769308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429ac89-ff16-447d-b5a9-ccf568ee6604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:57.770077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429ac89-ff16-447d-b5a9-ccf568ee6604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:57.774322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429ac89-ff16-447d-b5a9-ccf568ee6604 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:57.783323 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1274/2548] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:20:57.787346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:57.798969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62ebe7-730f-4835-adce-9c8df8a57776 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:57.804302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:57.805087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:57.805441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:57.805826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] PATH_INFO: `/v3/projects/b28579c61798424988aa02fe3a023294/users/6180f37cb0144f9f902305492aaa6f25/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:57.807894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28579c61798424988aa02fe3a023294, user_id=6180f37cb0144f9f902305492aaa6f25, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:57.810713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:57.831756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1275/2549] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:20:57 2022] PUT /identity/v3/projects/b28579c61798424988aa02fe3a023294/users/6180f37cb0144f9f902305492aaa6f25/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:20:57.833614 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f22f1-867c-4b7c-8603-808ebf3e6e9d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:57.833906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f22f1-867c-4b7c-8603-808ebf3e6e9d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:57.834182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f22f1-867c-4b7c-8603-808ebf3e6e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:57.863537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b17f22f1-867c-4b7c-8603-808ebf3e6e9d None None] MFA Rules not processed for user `6180f37cb0144f9f902305492aaa6f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:20:57.864890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b17f22f1-867c-4b7c-8603-808ebf3e6e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:57.946951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1275/2550] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:20:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:20:57.959137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:20:57.961038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:20:58.032271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353222ac-97a8-431f-8214-b8c784b7246b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:20:58.036791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8936de0-a433-4bc4-8ddc-3336a689022e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:20:58.037384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936de0-a433-4bc4-8ddc-3336a689022e service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:20:58.037705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936de0-a433-4bc4-8ddc-3336a689022e service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:20:58.037983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936de0-a433-4bc4-8ddc-3336a689022e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:20:58.102032 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8936de0-a433-4bc4-8ddc-3336a689022e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:20:58.103808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8936de0-a433-4bc4-8ddc-3336a689022e service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:20:58.114444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1276/2551] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:20:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:01.305901 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f22f1-867c-4b7c-8603-808ebf3e6e9d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:01.339869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:01.377885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f22f1-867c-4b7c-8603-808ebf3e6e9d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:01.384100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:01.384753 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:01.385031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:01.385447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] PATH_INFO: `/v3/users/56e206e4ad424e8e965635d82a50eb5f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:01.386627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e206e4ad424e8e965635d82a50eb5f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:01.388569 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:01.568134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e206e4ad424e8e965635d82a50eb5f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:01.569425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e206e4ad424e8e965635d82a50eb5f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:01.667387 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e206e4ad424e8e965635d82a50eb5f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:01.668079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e206e4ad424e8e965635d82a50eb5f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:01.682356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e206e4ad424e8e965635d82a50eb5f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:01.694997 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1276/2552] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:21:01 2022] DELETE /identity/v3/users/56e206e4ad424e8e965635d82a50eb5f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:01.699656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:01.700165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:01.700270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:01.719036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:01.720245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:01.778685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1277/2553] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:01.791406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:01.792840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:01.866214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced923-e36e-42d0-94e5-91c5d8dd2764 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:01.871944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:01.873175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:01.873175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:01.873396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:01.884849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:01.890518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:01.897959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1277/2554] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:02.344239 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:02.345840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:02.371572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6ec99-c099-42c6-b9fd-92a9321445a3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:02.371572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:02.371572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:02.371572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:02.371908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] PATH_INFO: `/v3/projects/305c3e7ed45f41a382c0e3344d34b361` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:02.371908 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305c3e7ed45f41a382c0e3344d34b361)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:02.372113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:02.375088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:02.508840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305c3e7ed45f41a382c0e3344d34b361', 'request_id': 'req-abf8907e-4863-462d-a96b-a464ba3dcda5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.515117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305c3e7ed45f41a382c0e3344d34b361', 'request_id': 'req-abf8907e-4863-462d-a96b-a464ba3dcda5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.524079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305c3e7ed45f41a382c0e3344d34b361', 'request_id': 'req-abf8907e-4863-462d-a96b-a464ba3dcda5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.624525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] The token cache is being invalidate because project 305c3e7ed45f41a382c0e3344d34b361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:21:02.624823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.629335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.629335 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1278/2555] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:21:02 2022] DELETE /identity/v3/projects/305c3e7ed45f41a382c0e3344d34b361 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:02.634063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:02.643629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ae05e-3de5-4751-b99a-760892cf15e5 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:02.691006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:02.705635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:02.708979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:02.708979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:02.708979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] PATH_INFO: `/v3/users/6180f37cb0144f9f902305492aaa6f25` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:02.708979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6180f37cb0144f9f902305492aaa6f25)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:02.710892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:02.916345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6180f37cb0144f9f902305492aaa6f25'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.917594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6180f37cb0144f9f902305492aaa6f25'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.972898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6180f37cb0144f9f902305492aaa6f25'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.973741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6180f37cb0144f9f902305492aaa6f25'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:02.992810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc242286-4d12-4913-b7e4-25f463e05715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6180f37cb0144f9f902305492aaa6f25'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:03.018845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1278/2556] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:21:02 2022] DELETE /identity/v3/users/6180f37cb0144f9f902305492aaa6f25 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:03.646018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:03.649193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:03.679465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8907e-4863-462d-a96b-a464ba3dcda5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:03.761998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:03.763077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:03.764055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:03.764512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] PATH_INFO: `/v3/projects/b28579c61798424988aa02fe3a023294` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:03.766429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28579c61798424988aa02fe3a023294)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:03.768831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:03.917869 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28579c61798424988aa02fe3a023294', 'request_id': 'req-cef94729-7844-423c-bbf5-06ee82dcb324'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:03.928429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28579c61798424988aa02fe3a023294', 'request_id': 'req-cef94729-7844-423c-bbf5-06ee82dcb324'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:03.942170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28579c61798424988aa02fe3a023294', 'request_id': 'req-cef94729-7844-423c-bbf5-06ee82dcb324'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:04.028247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] The token cache is being invalidate because project b28579c61798424988aa02fe3a023294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:21:04.028549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:04.030349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cef94729-7844-423c-bbf5-06ee82dcb324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:04.030917 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1279/2557] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:21:03 2022] DELETE /identity/v3/projects/b28579c61798424988aa02fe3a023294 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:04.035671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550205e0-ac78-4b8f-bdd9-ebf8f7d2c700 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.036105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550205e0-ac78-4b8f-bdd9-ebf8f7d2c700 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.036196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550205e0-ac78-4b8f-bdd9-ebf8f7d2c700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.057104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-550205e0-ac78-4b8f-bdd9-ebf8f7d2c700 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:04.059141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-550205e0-ac78-4b8f-bdd9-ebf8f7d2c700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:04.122646 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1279/2558] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:04.133752 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826b95-89dd-4cb3-b03c-d6d1257875ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.134124 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826b95-89dd-4cb3-b03c-d6d1257875ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.134488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826b95-89dd-4cb3-b03c-d6d1257875ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.155830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-21826b95-89dd-4cb3-b03c-d6d1257875ad None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:04.157072 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21826b95-89dd-4cb3-b03c-d6d1257875ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:04.169002 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1280/2559] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:04.173437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550205e0-ac78-4b8f-bdd9-ebf8f7d2c700 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:04.184920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550205e0-ac78-4b8f-bdd9-ebf8f7d2c700 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:04.190314 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:04.190948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.191219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.191491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.192138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:21:04.192433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:04.195062 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:04.222109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1280/2560] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:21:04.226263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826b95-89dd-4cb3-b03c-d6d1257875ad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:04.235567 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826b95-89dd-4cb3-b03c-d6d1257875ad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:04.240204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:04.241457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.241783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.242049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.242682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:04.244590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:04.259394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:04.333615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1281/2561] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:21:04 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:21:04.337051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:04.347671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1bef6-296c-48f1-8fee-b1b2b47111a9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:04.398220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:04.405363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:04.406023 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.406304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.406557 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.407202 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:04.409107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:04.505181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1281/2562] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:21:04 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:21:04.508177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:04.519015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c6127-64a6-4459-8550-ed32f427beb1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:04.524775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:04.525395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.525722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.526003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.526656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:04.528309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:04.538842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1282/2563] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:21:04.542883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:04.554164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65d2da-9564-4b4c-9337-0164c31c9533 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:04.559760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16514b-4f29-4680-b1a3-b394d34269db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:04.559760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16514b-4f29-4680-b1a3-b394d34269db admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.560067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16514b-4f29-4680-b1a3-b394d34269db admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.560957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16514b-4f29-4680-b1a3-b394d34269db admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.561837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16514b-4f29-4680-b1a3-b394d34269db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:04.563903 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16514b-4f29-4680-b1a3-b394d34269db admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:04.579489 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1282/2564] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:21:04.584882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:04.595632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52bcea-d21a-4b7b-aabc-01df9ef70348 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:04.600722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:04.601385 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.601832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.602121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] PATH_INFO: `/v3/projects/0d2f1b7a983e45e7be233e0dfa6f9e49/users/b3330c2adedc4b6db9eada1d64106beb/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.620326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d2f1b7a983e45e7be233e0dfa6f9e49, user_id=b3330c2adedc4b6db9eada1d64106beb, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:04.623526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:04.739192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1283/2565] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:21:04 2022] PUT /identity/v3/projects/0d2f1b7a983e45e7be233e0dfa6f9e49/users/b3330c2adedc4b6db9eada1d64106beb/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:21:04.750365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fb89d-3e5a-4c37-9df7-193b25c324fc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.750729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fb89d-3e5a-4c37-9df7-193b25c324fc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.750996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fb89d-3e5a-4c37-9df7-193b25c324fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.781170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6f2fb89d-3e5a-4c37-9df7-193b25c324fc None None] MFA Rules not processed for user `b3330c2adedc4b6db9eada1d64106beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:04.782468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6f2fb89d-3e5a-4c37-9df7-193b25c324fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:04.856220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1283/2566] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:21:04.868190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:04.870049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:04.929005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab5a19-42f1-4095-bd7b-668930b9c90b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:04.932883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9ab21-6c97-41ed-a3fa-29da7237f67c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:04.933488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ab21-6c97-41ed-a3fa-29da7237f67c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:04.933842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ab21-6c97-41ed-a3fa-29da7237f67c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:04.934123 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9ab21-6c97-41ed-a3fa-29da7237f67c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:04.944964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9ab21-6c97-41ed-a3fa-29da7237f67c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:04.946469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9ab21-6c97-41ed-a3fa-29da7237f67c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:04.955744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1284/2567] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:04.982099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fb89d-3e5a-4c37-9df7-193b25c324fc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:05.047918 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fb89d-3e5a-4c37-9df7-193b25c324fc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:05.054707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:05.055679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:05.055807 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:05.056169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] PATH_INFO: `/v3/users/b3330c2adedc4b6db9eada1d64106beb` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:05.057465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3330c2adedc4b6db9eada1d64106beb)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:05.059497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:05.295153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3330c2adedc4b6db9eada1d64106beb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:05.295790 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3330c2adedc4b6db9eada1d64106beb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:05.380600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3330c2adedc4b6db9eada1d64106beb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:05.381163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3330c2adedc4b6db9eada1d64106beb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:05.389702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:05.394149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3330c2adedc4b6db9eada1d64106beb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:05.406899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1284/2568] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:21:04 2022] DELETE /identity/v3/users/b3330c2adedc4b6db9eada1d64106beb => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:05.411376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:05.411694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:05.411993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:05.435923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:05.437117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:05.496611 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1285/2569] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:05.506263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:05.507789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:05.563937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15d37e-751e-4669-837c-8a7448bdf828 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:05.567513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8121224-74a0-4af2-ab13-ada02c3fd913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:05.568333 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8121224-74a0-4af2-ab13-ada02c3fd913 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:05.568622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8121224-74a0-4af2-ab13-ada02c3fd913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:05.568889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8121224-74a0-4af2-ab13-ada02c3fd913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:05.577635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8121224-74a0-4af2-ab13-ada02c3fd913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:05.579046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8121224-74a0-4af2-ab13-ada02c3fd913 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:05.588815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1285/2570] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:06.017820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:06.019278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:06.032171 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460d171-16b8-459e-8140-470d8c4904bf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:06.037252 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:06.038299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.042121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.042121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] PATH_INFO: `/v3/projects/0d2f1b7a983e45e7be233e0dfa6f9e49` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.042121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d2f1b7a983e45e7be233e0dfa6f9e49)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:06.042121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:06.179710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d2f1b7a983e45e7be233e0dfa6f9e49', 'request_id': 'req-e4651c08-fe88-45b6-a1ff-a002d400d260'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:06.183508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:06.185650 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d2f1b7a983e45e7be233e0dfa6f9e49', 'request_id': 'req-e4651c08-fe88-45b6-a1ff-a002d400d260'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:06.191853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d2f1b7a983e45e7be233e0dfa6f9e49', 'request_id': 'req-e4651c08-fe88-45b6-a1ff-a002d400d260'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:06.259412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] The token cache is being invalidate because project 0d2f1b7a983e45e7be233e0dfa6f9e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:21:06.259828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:06.260527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4651c08-fe88-45b6-a1ff-a002d400d260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:06.262093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1286/2571] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:21:06 2022] DELETE /identity/v3/projects/0d2f1b7a983e45e7be233e0dfa6f9e49 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:06.272159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e5f-e338-4adc-8900-688634ddf323 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.272911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e5f-e338-4adc-8900-688634ddf323 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.272911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e5f-e338-4adc-8900-688634ddf323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.297554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-186f7e5f-e338-4adc-8900-688634ddf323 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:06.298945 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-186f7e5f-e338-4adc-8900-688634ddf323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:06.370994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1286/2572] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:06.378680 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccddfc9-00fd-4df7-a12f-654aea315eba None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.378971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccddfc9-00fd-4df7-a12f-654aea315eba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.379248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccddfc9-00fd-4df7-a12f-654aea315eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.399392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2ccddfc9-00fd-4df7-a12f-654aea315eba None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:06.400580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ccddfc9-00fd-4df7-a12f-654aea315eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:06.408377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1287/2573] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:06.412588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7e5f-e338-4adc-8900-688634ddf323 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:06.422348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7e5f-e338-4adc-8900-688634ddf323 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:06.426929 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:06.427522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.428318 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.428779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.429539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:21:06.429883 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:06.431477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:06.442822 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1287/2574] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:21:06.447771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccddfc9-00fd-4df7-a12f-654aea315eba None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:06.456971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccddfc9-00fd-4df7-a12f-654aea315eba None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:06.462735 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:06.463331 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.463612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.463933 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.464615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:06.466564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:06.581042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1288/2575] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:21:06 2022] POST /identity/v3/projects => generated 391 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:21:06.585655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:06.595604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21dce-9941-46b8-b9ea-b408bc3532eb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:06.656970 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:06.658376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.658376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.658376 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.659100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:06.660998 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:06.723653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1288/2576] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:21:06 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:21:06.728766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:06.739467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9630ce-6cc2-4ddd-acdd-2423904a21c8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:06.744828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:06.745532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.745850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.746127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.746772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:06.748412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:06.762645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1289/2577] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:21:06.763882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:06.770689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:06.785628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0bd1e-cd83-4c52-ac5f-f1b6b0ce1925 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:06.785628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03543118-f3e9-4e72-8756-99d5fd77040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:06.785628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03543118-f3e9-4e72-8756-99d5fd77040a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.785628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03543118-f3e9-4e72-8756-99d5fd77040a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.786243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03543118-f3e9-4e72-8756-99d5fd77040a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.786243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03543118-f3e9-4e72-8756-99d5fd77040a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:06.786243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03543118-f3e9-4e72-8756-99d5fd77040a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:06.796451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1289/2578] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:21:06.800739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:06.811432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea4753-26f3-4c2f-8889-fd800031c08f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:06.816233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:06.816890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.817175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.817457 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] PATH_INFO: `/v3/projects/881c9178d52d490fbeca130f07d4f362/users/c5a1db4af03248fdaa22f32f823551b0/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:06.832815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=881c9178d52d490fbeca130f07d4f362, user_id=c5a1db4af03248fdaa22f32f823551b0, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:06.836177 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:06.921790 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1290/2579] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:21:06 2022] PUT /identity/v3/projects/881c9178d52d490fbeca130f07d4f362/users/c5a1db4af03248fdaa22f32f823551b0/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:21:06.988664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf4c9-45d3-4900-9294-9d384a7b6912 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:06.988948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf4c9-45d3-4900-9294-9d384a7b6912 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:06.989225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf4c9-45d3-4900-9294-9d384a7b6912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:07.020002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a6bdf4c9-45d3-4900-9294-9d384a7b6912 None None] MFA Rules not processed for user `c5a1db4af03248fdaa22f32f823551b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:07.021241 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a6bdf4c9-45d3-4900-9294-9d384a7b6912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:07.086317 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1290/2580] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:07.097025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:07.098856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:07.154294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215331df-7d98-4444-a864-424f2b5fd55f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:07.157928 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:07.158517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:07.158795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:07.159067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:07.167288 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:07.168717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:07.177188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1291/2581] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:12.529631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf9d0f-a3d6-4261-9351-e7dae39db82d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:12.530157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf9d0f-a3d6-4261-9351-e7dae39db82d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:12.530157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf9d0f-a3d6-4261-9351-e7dae39db82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:12.558128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-edaf9d0f-a3d6-4261-9351-e7dae39db82d None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:12.560673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-edaf9d0f-a3d6-4261-9351-e7dae39db82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:12.632097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1291/2582] 10.222.0.39 () {58 vars in 967 bytes} [Sun Sep 4 16:21:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:12.646210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:12.648972 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:12.733696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407d00d-0aad-41c1-a00c-bcd04b4ffd08 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:12.738953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:12.739571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:12.739856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:12.740135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:12.752502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:12.754142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:12.763602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1292/2583] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:21:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:21:18.339268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:18.339651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:18.340018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:18.360016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:18.361252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:18.368718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1292/2584] 10.222.0.39 () {58 vars in 967 bytes} [Sun Sep 4 16:21:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:18.380965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:18.391421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cffaa5-1019-48dc-9a45-47d49f9ab5d5 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:18.396506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:18.397097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:18.397376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:18.397712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:18.406947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:18.408573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:18.412379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:18.418689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1293/2585] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:21:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:21:27.073084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:27.074513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:27.144490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669320fd-e624-4b5c-a6eb-b7a81306df06 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:27.150339 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:27.150957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:27.151240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:27.151516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:27.153379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:27.164973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:27.166656 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:27.177150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1293/2586] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:27.203372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:27.204903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:27.285010 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c3507-40d8-4248-b836-59bcad1b9c90 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:27.290420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:27.291036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:27.291317 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:27.291590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] PATH_INFO: `/v3/users/9b17d0d51b034064a98bdae4bbc20fb6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:27.292722 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b17d0d51b034064a98bdae4bbc20fb6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:27.294840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:27.516471 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b17d0d51b034064a98bdae4bbc20fb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:27.517133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b17d0d51b034064a98bdae4bbc20fb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:27.530264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b17d0d51b034064a98bdae4bbc20fb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:27.619203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b17d0d51b034064a98bdae4bbc20fb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:27.619848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b17d0d51b034064a98bdae4bbc20fb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:27.632665 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1294/2587] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:21:27 2022] DELETE /identity/v3/users/9b17d0d51b034064a98bdae4bbc20fb6 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:27.866313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:27.879313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87f4bf-383e-44a5-bb80-1c771a6935b2 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:27.884271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:27.942835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:27.942835 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:27.943576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:27.943576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] PATH_INFO: `/v3/projects/ae092c3124e9438fa20dfda624775825` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:27.944534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae092c3124e9438fa20dfda624775825)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:27.946562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:28.063213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae092c3124e9438fa20dfda624775825', 'request_id': 'req-ea42615b-5ea5-493f-a719-b998d14874e1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.119162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae092c3124e9438fa20dfda624775825', 'request_id': 'req-ea42615b-5ea5-493f-a719-b998d14874e1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.124858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae092c3124e9438fa20dfda624775825', 'request_id': 'req-ea42615b-5ea5-493f-a719-b998d14874e1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.135943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] The token cache is being invalidate because project ae092c3124e9438fa20dfda624775825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:21:28.136238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.136869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.138380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1294/2588] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:21:27 2022] DELETE /identity/v3/projects/ae092c3124e9438fa20dfda624775825 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:28.142442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:28.143888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:28.157685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d3a77-554a-4c01-8f3f-ed4cc966d1a8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:28.213635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:28.214258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:28.214571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:28.214854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] PATH_INFO: `/v3/users/7e6f70cd24b7424ab0cf94c494ebdcbc` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:28.215987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6f70cd24b7424ab0cf94c494ebdcbc)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:28.218104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:28.238120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:28.334257 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:28.335909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:28.351934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42615b-5ea5-493f-a719-b998d14874e1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:28.356018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:28.356649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:28.356937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:28.357219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] PATH_INFO: `/v3/users/c5a1db4af03248fdaa22f32f823551b0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:28.358400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a1db4af03248fdaa22f32f823551b0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:28.360337 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:28.425005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6f70cd24b7424ab0cf94c494ebdcbc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.425443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6f70cd24b7424ab0cf94c494ebdcbc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.437360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6f70cd24b7424ab0cf94c494ebdcbc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.527575 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6f70cd24b7424ab0cf94c494ebdcbc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.528197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8937754-ce36-4a9d-9d5d-3551397711e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6f70cd24b7424ab0cf94c494ebdcbc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.541822 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1295/2589] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:21:28 2022] DELETE /identity/v3/users/7e6f70cd24b7424ab0cf94c494ebdcbc => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:28.550524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a1db4af03248fdaa22f32f823551b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.551280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a1db4af03248fdaa22f32f823551b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.616065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a1db4af03248fdaa22f32f823551b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.616760 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a1db4af03248fdaa22f32f823551b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.629975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a1db4af03248fdaa22f32f823551b0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.644297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1295/2590] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:21:28 2022] DELETE /identity/v3/users/c5a1db4af03248fdaa22f32f823551b0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:21:28.649073 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:28.649656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:28.649914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:28.669761 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:21:28.670975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:28.731143 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1296/2591] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:21:28.740789 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:28.742215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:28.756289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:28.778746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:28.780238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:28.793652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e18cf-0a11-490f-9481-adb661b04e0c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:28.798390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:28.799022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:28.799300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:28.799568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] PATH_INFO: `/v3/projects/9e40d38756e04664a630bfc16c9d411d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:28.800695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e40d38756e04664a630bfc16c9d411d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:28.802662 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:28.804623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3232271-40d4-493c-b2ca-5abc0e7088f5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:28.810545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:28.810545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:28.814849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:28.814849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:28.822612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:28.824197 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:28.834794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1296/2592] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:28.909764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e40d38756e04664a630bfc16c9d411d', 'request_id': 'req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.916410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e40d38756e04664a630bfc16c9d411d', 'request_id': 'req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:28.922810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e40d38756e04664a630bfc16c9d411d', 'request_id': 'req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.016181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] The token cache is being invalidate because project 9e40d38756e04664a630bfc16c9d411d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:21:29.016498 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.017216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.018784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1297/2593] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:21:28 2022] DELETE /identity/v3/projects/9e40d38756e04664a630bfc16c9d411d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:29.321153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:29.323251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:29.336794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17645a49-4ae7-4a13-b1d5-4c375578c593 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:29.396738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:29.397382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:29.397675 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:29.397949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] PATH_INFO: `/v3/projects/881c9178d52d490fbeca130f07d4f362` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:29.399201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=881c9178d52d490fbeca130f07d4f362)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:29.401983 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:29.505769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881c9178d52d490fbeca130f07d4f362', 'request_id': 'req-408c9186-5522-4dda-b1d9-88b1f041d864'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.567588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881c9178d52d490fbeca130f07d4f362', 'request_id': 'req-408c9186-5522-4dda-b1d9-88b1f041d864'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.574549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881c9178d52d490fbeca130f07d4f362', 'request_id': 'req-408c9186-5522-4dda-b1d9-88b1f041d864'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.585828 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] The token cache is being invalidate because project 881c9178d52d490fbeca130f07d4f362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:21:29.586124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.586800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:21:29.588538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1297/2594] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:21:29 2022] DELETE /identity/v3/projects/881c9178d52d490fbeca130f07d4f362 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:21:39.012750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=15,keystone:SELECT=74,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:39.582733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:51.305770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:51.307657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:51.384671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:51.392780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:51.501543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab88e4-c16f-4a91-9e50-b996c3e4ad7a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:51.507903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:51.508727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:51.509299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:51.509792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:51.512242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:51.518022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c9186-5522-4dda-b1d9-88b1f041d864 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:51.528023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:51.530178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:51.532402 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:51.533315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:51.533729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:51.534100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:51.543170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1298/2595] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:51.552156 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:51.554087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:51.566721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1298/2596] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:21:56.965306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:56.965306 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:57.026903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24156-1552-4fc5-a74a-35815b8e27e6 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:57.030651 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:57.031245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:57.031525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:57.031797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:57.033492 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:57.068625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:21:57.118824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:57.120295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:57.141411 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1299/2597] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:21:58.379111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:21:58.380668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:58.395693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ca388-299f-418c-845f-8dcc2ebe4f50 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:21:58.399663 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:21:58.400647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:21:58.401047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:21:58.401422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:21:58.403586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:21:58.418028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:21:58.419592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:21:58.432312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1299/2598] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:01.177165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:01.178776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:01.193528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24fe44-2550-4ca8-acb7-48475ce800e4 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:01.198704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:01.199424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:01.199767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:01.200136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] PATH_INFO: `/v3/users/1623e93ad6844cacae7cc1a71b339f22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:01.201431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1623e93ad6844cacae7cc1a71b339f22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:01.203544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:01.436789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1623e93ad6844cacae7cc1a71b339f22'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:01.437510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1623e93ad6844cacae7cc1a71b339f22'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:01.459988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1623e93ad6844cacae7cc1a71b339f22'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:01.554556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1623e93ad6844cacae7cc1a71b339f22'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:01.555378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f07a2f50-88fe-4d41-bf1e-5ee4213c880f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1623e93ad6844cacae7cc1a71b339f22'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:01.577454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1300/2599] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:01 2022] DELETE /identity/v3/users/1623e93ad6844cacae7cc1a71b339f22 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:01.890499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:01.901730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112635a4-1136-4ca3-b71a-d550dd505cd3 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:01.973820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:01.974699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:01.975093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:01.975470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] PATH_INFO: `/v3/projects/3219dc7400ec4957a31b1b0fabfd8c56` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:01.976888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3219dc7400ec4957a31b1b0fabfd8c56)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:01.979024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.108629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3219dc7400ec4957a31b1b0fabfd8c56', 'request_id': 'req-c65ec2d4-322b-4029-a9d3-52ba677f3489'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.113442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:02.185150 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3219dc7400ec4957a31b1b0fabfd8c56', 'request_id': 'req-c65ec2d4-322b-4029-a9d3-52ba677f3489'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.192168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3219dc7400ec4957a31b1b0fabfd8c56', 'request_id': 'req-c65ec2d4-322b-4029-a9d3-52ba677f3489'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.209420 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] The token cache is being invalidate because project 3219dc7400ec4957a31b1b0fabfd8c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:02.209751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.210478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c65ec2d4-322b-4029-a9d3-52ba677f3489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.212185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1300/2600] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:01 2022] DELETE /identity/v3/projects/3219dc7400ec4957a31b1b0fabfd8c56 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:02.221685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3cde-aaab-41d7-92e1-3bba3c05f46a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.221685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3cde-aaab-41d7-92e1-3bba3c05f46a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.221685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3cde-aaab-41d7-92e1-3bba3c05f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.285865 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-278b3cde-aaab-41d7-92e1-3bba3c05f46a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:02.287213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-278b3cde-aaab-41d7-92e1-3bba3c05f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:02.353093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:02.379899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1301/2601] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:02.393396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefef04-8930-440a-90fd-9b0705872fe5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.393994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefef04-8930-440a-90fd-9b0705872fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.397863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefef04-8930-440a-90fd-9b0705872fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.420892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ffefef04-8930-440a-90fd-9b0705872fe5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:02.422276 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ffefef04-8930-440a-90fd-9b0705872fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:02.432134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1301/2602] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:02.436807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3cde-aaab-41d7-92e1-3bba3c05f46a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:02.448265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3cde-aaab-41d7-92e1-3bba3c05f46a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:02.462140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:02.462715 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.462993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.463365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.464068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:22:02.464440 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:02.466104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.480318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1302/2603] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:22:02.485893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefef04-8930-440a-90fd-9b0705872fe5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:02.497203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefef04-8930-440a-90fd-9b0705872fe5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:02.503155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:02.503846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.504201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.504552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.505315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:02.507395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.593034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:02.594967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:02.614699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff6b9b-7ed4-4a81-8fd6-186bd43bfc17 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:02.617188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1302/2604] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:02 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:02.620766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:02.621400 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.621714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.621985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] PATH_INFO: `/v3/users/6e5c22b631df4600b9f920120b67fc67` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.622718 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:02.623175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e5c22b631df4600b9f920120b67fc67)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:02.625098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.686515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615db33-c97d-4500-90d7-aa92fe5c2e56 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:02.691632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:02.692240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.692551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.692827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.693471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:02.695837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.805886 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1303/2605] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:02 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:22:02.809912 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:02.820423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b20cc-f785-410f-aa2f-6f1782d8bed8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:02.826591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:02.828899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.828899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.828899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.828899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:02.830345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.835190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e5c22b631df4600b9f920120b67fc67'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.835856 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e5c22b631df4600b9f920120b67fc67'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.840579 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1304/2606] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:22:02.845308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:02.847363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:02.850082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e5c22b631df4600b9f920120b67fc67'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.860021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a689c-842e-42c9-b023-8102cc5bd340 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:02.864829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:02.865425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.865775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.868982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.868982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:02.868982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.879080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1305/2607] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:22:02.883222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:02.893590 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b46cb3-f43c-4113-8428-f4a8a8b04821 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:02.898479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:02.899148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:02.899430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:02.899708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] PATH_INFO: `/v3/projects/6dde43a19e95472d87ea4273922156be/users/b1fa7bf738264e77b868ce9d7863214e/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:02.915262 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dde43a19e95472d87ea4273922156be, user_id=b1fa7bf738264e77b868ce9d7863214e, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:02.916609 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e5c22b631df4600b9f920120b67fc67'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.917301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d56c9f6d-3a64-4393-a19d-db89830184fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e5c22b631df4600b9f920120b67fc67'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:02.917948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:02.930055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1303/2608] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:02 2022] DELETE /identity/v3/users/6e5c22b631df4600b9f920120b67fc67 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:02.986214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1306/2609] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:02 2022] PUT /identity/v3/projects/6dde43a19e95472d87ea4273922156be/users/b1fa7bf738264e77b868ce9d7863214e/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:03.001369 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.001804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.002146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.023319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:03.024678 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:03.093394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1304/2610] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:03.102909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:03.104655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:03.171279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:03.186477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41257fc-05ba-4667-add3-06c1f277e8f6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:03.190381 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:03.192408 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:03.192607 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d7ca-08e5-45b9-abb3-ac826cd1ff7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:03.193282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d7ca-08e5-45b9-abb3-ac826cd1ff7b service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.193597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d7ca-08e5-45b9-abb3-ac826cd1ff7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.194082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d7ca-08e5-45b9-abb3-ac826cd1ff7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.204673 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9d7ca-08e5-45b9-abb3-ac826cd1ff7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:03.206157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9d7ca-08e5-45b9-abb3-ac826cd1ff7b service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:03.207197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8a881-c5a9-4fa1-a825-423847aa23a3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:03.212598 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:03.213909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.214362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.220375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] PATH_INFO: `/v3/projects/1cc5ea4158344d5c8a35bc5783410620` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.220375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc5ea4158344d5c8a35bc5783410620)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:03.220375 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:03.221237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1307/2611] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:03.373374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:03.383468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc5ea4158344d5c8a35bc5783410620', 'request_id': 'req-d0da88da-36f6-431f-bec7-86585df37c05'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:03.390349 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc5ea4158344d5c8a35bc5783410620', 'request_id': 'req-d0da88da-36f6-431f-bec7-86585df37c05'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:03.397353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc5ea4158344d5c8a35bc5783410620', 'request_id': 'req-d0da88da-36f6-431f-bec7-86585df37c05'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:03.476316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] The token cache is being invalidate because project 1cc5ea4158344d5c8a35bc5783410620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:03.476610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:03.477291 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d0da88da-36f6-431f-bec7-86585df37c05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:03.478941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1305/2612] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:03 2022] DELETE /identity/v3/projects/1cc5ea4158344d5c8a35bc5783410620 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:03.484180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288fb8e-8067-4a0a-a2e9-555473c6e896 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.484469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288fb8e-8067-4a0a-a2e9-555473c6e896 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.484747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288fb8e-8067-4a0a-a2e9-555473c6e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.504366 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d288fb8e-8067-4a0a-a2e9-555473c6e896 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:03.505593 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d288fb8e-8067-4a0a-a2e9-555473c6e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:03.569221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1308/2613] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:22:03.579263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7819a8d-37b3-4154-a280-2878e9326007 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.579263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7819a8d-37b3-4154-a280-2878e9326007 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.579263 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7819a8d-37b3-4154-a280-2878e9326007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.599303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d7819a8d-37b3-4154-a280-2878e9326007 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:03.600538 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d7819a8d-37b3-4154-a280-2878e9326007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:03.609911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1306/2614] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:03.614293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288fb8e-8067-4a0a-a2e9-555473c6e896 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:03.624700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288fb8e-8067-4a0a-a2e9-555473c6e896 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:03.629370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:03.630004 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.630288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.630561 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.631204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:22:03.631508 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:03.633085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:03.645466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1309/2615] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:22:03.650017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7819a8d-37b3-4154-a280-2878e9326007 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:03.659944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7819a8d-37b3-4154-a280-2878e9326007 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:03.667074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:03.667795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.668082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.668359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.669065 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:03.671597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:03.778978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1307/2616] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:03 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:03.781037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:03.791155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325125e-9b8f-4cc1-adea-002395f2fb1a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:03.852515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:03.853161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.853406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.853715 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.854360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:03.856330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:03.963195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1310/2617] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:03 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:22:03.968412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:03.978853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74f076-b1f7-4773-bca8-9541394d41e3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:03.984734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:03.985488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:03.985986 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:03.986365 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:03.987115 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:03.988732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:04.002289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1308/2618] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:04.003555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:04.014215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d4b8a-2063-4e4c-bfef-d4e7c2bfa116 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:04.019944 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09355635-dc31-4830-a632-f20e37a948d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:04.020548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09355635-dc31-4830-a632-f20e37a948d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:04.020827 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09355635-dc31-4830-a632-f20e37a948d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:04.021100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09355635-dc31-4830-a632-f20e37a948d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:04.021769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09355635-dc31-4830-a632-f20e37a948d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:04.023360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09355635-dc31-4830-a632-f20e37a948d7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:04.035486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1311/2619] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:04.040139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:04.051701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b9d3-8b3a-42f3-ba35-8ae6337f95ba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:04.056555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:04.057224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:04.057504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:04.057819 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] PATH_INFO: `/v3/projects/cf5f20da263e45abb540fa062ab9aa19/users/6e6309d0ca0b452f8e05632349fb17d6/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:04.073453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5f20da263e45abb540fa062ab9aa19, user_id=6e6309d0ca0b452f8e05632349fb17d6, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:04.076679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:04.188714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1309/2620] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:04 2022] PUT /identity/v3/projects/cf5f20da263e45abb540fa062ab9aa19/users/6e6309d0ca0b452f8e05632349fb17d6/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:04.200136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecddad-72e2-473e-9d6a-c38a6db081ff None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:04.200428 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecddad-72e2-473e-9d6a-c38a6db081ff None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:04.200713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecddad-72e2-473e-9d6a-c38a6db081ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:04.229875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b6ecddad-72e2-473e-9d6a-c38a6db081ff None None] MFA Rules not processed for user `6e6309d0ca0b452f8e05632349fb17d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:04.231092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b6ecddad-72e2-473e-9d6a-c38a6db081ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:04.269904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:04.302240 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1312/2621] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:04.314103 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:04.315899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:04.421554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5940a4-6be4-4754-b15c-0e20e3c2b64f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:04.427393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:04.427998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:04.428286 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:04.428568 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:04.437038 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:04.440551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:04.447748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1310/2622] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:08.790427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f47e-22be-4b93-b024-119c5a710b6f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:08.790982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f47e-22be-4b93-b024-119c5a710b6f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:08.791198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f47e-22be-4b93-b024-119c5a710b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:08.822609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5387f47e-22be-4b93-b024-119c5a710b6f None None] MFA Rules not processed for user `b1fa7bf738264e77b868ce9d7863214e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:08.824312 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5387f47e-22be-4b93-b024-119c5a710b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:08.894570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1313/2623] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:08.911347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:08.923772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c0065-c851-423d-a174-2936f1f9a805 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:08.929713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:08.930324 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:08.930606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:08.930883 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:08.940489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:08.942265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:08.952037 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1311/2624] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:11.376559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9501ff-8009-426a-b2f2-d2dab279296a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:11.382884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9501ff-8009-426a-b2f2-d2dab279296a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:11.382884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9501ff-8009-426a-b2f2-d2dab279296a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:11.403782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1d9501ff-8009-426a-b2f2-d2dab279296a None None] MFA Rules not processed for user `b1fa7bf738264e77b868ce9d7863214e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:11.405002 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1d9501ff-8009-426a-b2f2-d2dab279296a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:11.417511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1314/2625] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:12.875581 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:12.884272 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:12.944179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdd3ea-01cf-45d7-a363-8d318afc39a1 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:12.948886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:12.949528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:12.949831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:12.950106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] PATH_INFO: `/v3/users/6e6309d0ca0b452f8e05632349fb17d6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:12.951249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6309d0ca0b452f8e05632349fb17d6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:12.953437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:13.180444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6309d0ca0b452f8e05632349fb17d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:13.181102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6309d0ca0b452f8e05632349fb17d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:13.194229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6309d0ca0b452f8e05632349fb17d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:13.256757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6309d0ca0b452f8e05632349fb17d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:13.257441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6309d0ca0b452f8e05632349fb17d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:13.274287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1312/2626] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:12 2022] DELETE /identity/v3/users/6e6309d0ca0b452f8e05632349fb17d6 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:13.279094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826968b-aa8c-4797-839e-026402a94910 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:13.279617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826968b-aa8c-4797-839e-026402a94910 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:13.279617 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826968b-aa8c-4797-839e-026402a94910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:13.299962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d826968b-aa8c-4797-839e-026402a94910 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:13.301185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d826968b-aa8c-4797-839e-026402a94910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:13.362498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1315/2627] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:13.372888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:13.375308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:13.434444 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa2766-0dd9-442b-91bb-1a1c6a6392c1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:13.438675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21338007-df57-4150-bf61-a20da26a63fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:13.439279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338007-df57-4150-bf61-a20da26a63fb service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:13.439560 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338007-df57-4150-bf61-a20da26a63fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:13.439837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338007-df57-4150-bf61-a20da26a63fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:13.448963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21338007-df57-4150-bf61-a20da26a63fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:13.451485 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21338007-df57-4150-bf61-a20da26a63fb service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:13.460398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1313/2628] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:13.892534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826968b-aa8c-4797-839e-026402a94910 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:13.894749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d826968b-aa8c-4797-839e-026402a94910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:13.908348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826968b-aa8c-4797-839e-026402a94910 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:13.913099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:13.913792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:13.914081 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:13.914361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] PATH_INFO: `/v3/projects/cf5f20da263e45abb540fa062ab9aa19` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:13.915545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5f20da263e45abb540fa062ab9aa19)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:13.917664 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:14.075514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5f20da263e45abb540fa062ab9aa19', 'request_id': 'req-34e9cf94-5010-43e3-8a07-d0bca8a7df85'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:14.080068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:14.157546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5f20da263e45abb540fa062ab9aa19', 'request_id': 'req-34e9cf94-5010-43e3-8a07-d0bca8a7df85'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:14.163505 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5f20da263e45abb540fa062ab9aa19', 'request_id': 'req-34e9cf94-5010-43e3-8a07-d0bca8a7df85'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:14.175861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] The token cache is being invalidate because project cf5f20da263e45abb540fa062ab9aa19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:14.176155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:14.176873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-34e9cf94-5010-43e3-8a07-d0bca8a7df85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:14.178417 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1316/2629] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:13 2022] DELETE /identity/v3/projects/cf5f20da263e45abb540fa062ab9aa19 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:14.183505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4fd5f-1293-4965-a1e9-db9017c612d6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.184054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4fd5f-1293-4965-a1e9-db9017c612d6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.184054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4fd5f-1293-4965-a1e9-db9017c612d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.204628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f5e4fd5f-1293-4965-a1e9-db9017c612d6 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:14.206054 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f5e4fd5f-1293-4965-a1e9-db9017c612d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:14.236413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:14.276808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1314/2630] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:14.290262 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dc2a1-7612-47ee-a5e0-0ee47613a6fb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.290800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dc2a1-7612-47ee-a5e0-0ee47613a6fb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.291170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dc2a1-7612-47ee-a5e0-0ee47613a6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.319534 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f30dc2a1-7612-47ee-a5e0-0ee47613a6fb None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:14.321058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f30dc2a1-7612-47ee-a5e0-0ee47613a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:14.336464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1317/2631] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:14.342930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4fd5f-1293-4965-a1e9-db9017c612d6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:14.349092 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4fd5f-1293-4965-a1e9-db9017c612d6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:14.356483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:14.357101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.357377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.357879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.358776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:22:14.359604 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:14.361670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:14.386670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1315/2632] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 16:22:14.395379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dc2a1-7612-47ee-a5e0-0ee47613a6fb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:14.404074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dc2a1-7612-47ee-a5e0-0ee47613a6fb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:14.410052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:14.410978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.411592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.412061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.412963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:14.415287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:14.517652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1318/2633] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:14 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:14.522741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:14.534969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b0c86-0e11-4c40-81a0-adbad4c21cfe admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:14.618624 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:14.620074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.620074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.620074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.620539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:14.622601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:14.727795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1316/2634] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:14 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:14.732065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:14.746572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d688f-49a4-4dc1-8e9e-3c69fce39287 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:14.756334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:14.757187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.758634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.758634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.758634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:14.760153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:14.775157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1319/2635] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:22:14.778327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:14.790701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392b7b7-f35d-4d0f-af32-e9f5cc403082 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:14.798826 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6f4c-d5a3-41ec-a04a-d8ec2b09180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:14.799880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6f4c-d5a3-41ec-a04a-d8ec2b09180f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.800353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6f4c-d5a3-41ec-a04a-d8ec2b09180f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.800841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6f4c-d5a3-41ec-a04a-d8ec2b09180f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.802086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6f4c-d5a3-41ec-a04a-d8ec2b09180f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:14.804338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6f4c-d5a3-41ec-a04a-d8ec2b09180f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:14.819620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1317/2636] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:22:14.824437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:14.836773 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01e29-fac0-4b88-8cd6-079cfe0f2618 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:14.842379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:14.843085 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.843399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.843846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] PATH_INFO: `/v3/projects/ebc32985b5fb4372b8f0764d8d5f1e4c/users/48ebcbe7fce84a52ac085979d96e1098/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:14.864792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc32985b5fb4372b8f0764d8d5f1e4c, user_id=48ebcbe7fce84a52ac085979d96e1098, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:14.869096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:14.973594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1320/2637] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:14 2022] PUT /identity/v3/projects/ebc32985b5fb4372b8f0764d8d5f1e4c/users/48ebcbe7fce84a52ac085979d96e1098/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:14.986744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:14.988081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:14.988419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:15.016844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:15.020496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:15.027944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] MFA Rules not processed for user `48ebcbe7fce84a52ac085979d96e1098`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:15.029386 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:15.117791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c519e-b46d-43d7-9b23-d52084bb8d51 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:15.121923 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1318/2638] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:15.125071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:15.126423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:15.127035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:15.127316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:15.130695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:15.131420 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:15.132898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:15.181505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:15.220160 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:15.221730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:15.228737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be590b5-c0b6-40d1-a89e-255d753bc2c9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:15.231736 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1321/2639] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:15.234873 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f59e24-bfbb-4a04-b2bb-697d54e68194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:15.235572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f59e24-bfbb-4a04-b2bb-697d54e68194 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:15.236018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f59e24-bfbb-4a04-b2bb-697d54e68194 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:15.236319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f59e24-bfbb-4a04-b2bb-697d54e68194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:15.244727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f59e24-bfbb-4a04-b2bb-697d54e68194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:15.246465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f59e24-bfbb-4a04-b2bb-697d54e68194 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:15.256764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1319/2640] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:15.518069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:15.556953 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:15.589633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e106cc-48d4-4fe6-8f46-3a35959f7a36 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:15.595256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:15.595937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:15.596207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:15.596492 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] PATH_INFO: `/v3/users/48ebcbe7fce84a52ac085979d96e1098` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:15.597702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ebcbe7fce84a52ac085979d96e1098)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:15.599689 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:15.891594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ebcbe7fce84a52ac085979d96e1098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:15.895431 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ebcbe7fce84a52ac085979d96e1098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:15.994874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ebcbe7fce84a52ac085979d96e1098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:15.995247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ebcbe7fce84a52ac085979d96e1098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:16.011957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ebcbe7fce84a52ac085979d96e1098'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:16.042236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1322/2641] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:15 2022] DELETE /identity/v3/users/48ebcbe7fce84a52ac085979d96e1098 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:16.044851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:16.045352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:16.045676 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:16.065872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:16.067109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:16.145930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1320/2642] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:16.155354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:16.157147 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:16.230059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481b30-2434-4e96-977e-29b18a982935 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:16.235033 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9ee43-f8ac-40b3-a030-c81a2338c676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:16.235775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9ee43-f8ac-40b3-a030-c81a2338c676 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:16.236466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9ee43-f8ac-40b3-a030-c81a2338c676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:16.236466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9ee43-f8ac-40b3-a030-c81a2338c676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:16.250951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9ee43-f8ac-40b3-a030-c81a2338c676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:16.252488 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9ee43-f8ac-40b3-a030-c81a2338c676 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:16.264315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1323/2643] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:16.858621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:16.861544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:16.886069 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fe853-5016-4191-9e63-8dcab8954d75 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:16.891936 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:16.892787 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:16.893137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:16.893484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] PATH_INFO: `/v3/projects/ebc32985b5fb4372b8f0764d8d5f1e4c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:16.895175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc32985b5fb4372b8f0764d8d5f1e4c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:16.897888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:17.007056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc32985b5fb4372b8f0764d8d5f1e4c', 'request_id': 'req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:17.014801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc32985b5fb4372b8f0764d8d5f1e4c', 'request_id': 'req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:17.022559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc32985b5fb4372b8f0764d8d5f1e4c', 'request_id': 'req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:17.097557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] The token cache is being invalidate because project ebc32985b5fb4372b8f0764d8d5f1e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:17.102289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:17.102289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-58b70d11-8c7a-4b2e-85b5-1afbcb827f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:17.102289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1321/2644] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:16 2022] DELETE /identity/v3/projects/ebc32985b5fb4372b8f0764d8d5f1e4c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:17.105203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c122-5fd4-40b2-be0d-df660c8d569b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.106400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c122-5fd4-40b2-be0d-df660c8d569b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.106916 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c122-5fd4-40b2-be0d-df660c8d569b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.134624 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3607c122-5fd4-40b2-be0d-df660c8d569b None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:17.136308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3607c122-5fd4-40b2-be0d-df660c8d569b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:17.198927 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:17.211473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1324/2645] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:17.219398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299cac2-87cb-4f9b-9f3f-49634ba81220 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.219696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299cac2-87cb-4f9b-9f3f-49634ba81220 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.219981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299cac2-87cb-4f9b-9f3f-49634ba81220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.241435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6299cac2-87cb-4f9b-9f3f-49634ba81220 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:17.242691 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6299cac2-87cb-4f9b-9f3f-49634ba81220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:17.251772 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1322/2646] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:17.256149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c122-5fd4-40b2-be0d-df660c8d569b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:17.265913 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c122-5fd4-40b2-be0d-df660c8d569b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:17.270601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:17.271214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.271499 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.271778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.272434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:22:17.272738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:17.274385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:17.289477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1325/2647] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:22:17.293998 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299cac2-87cb-4f9b-9f3f-49634ba81220 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:17.304872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299cac2-87cb-4f9b-9f3f-49634ba81220 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:17.309666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:17.310346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.310785 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.311031 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.311731 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:17.314284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:17.396119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1323/2648] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:17 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:17.400461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:17.411206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beff62-f9c1-4ba3-9fcd-31bfd672c6d7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:17.472645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:17.473268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.473553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.473888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.474518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:17.476399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:17.588050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1326/2649] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:17 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:17.592213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:17.602798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10092d0-91a3-4901-879f-fc393f1e048a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:17.607431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:17.608026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.608305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.608578 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.609421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:17.611056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:17.615327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:17.621218 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1324/2650] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:17.625707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:17.635568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6e4c6-5bda-439c-9268-a1451c17e33b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:17.640145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4204fb-8151-4354-a8ca-14734ff6c794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:17.641419 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4204fb-8151-4354-a8ca-14734ff6c794 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.641681 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4204fb-8151-4354-a8ca-14734ff6c794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.641959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4204fb-8151-4354-a8ca-14734ff6c794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.642591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4204fb-8151-4354-a8ca-14734ff6c794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:17.644159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4204fb-8151-4354-a8ca-14734ff6c794 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:17.654112 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1327/2651] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:17.658564 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:17.668983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57f47-6893-477b-8e6f-87bfcc4d0906 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:17.673775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:17.674426 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.674705 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.674978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] PATH_INFO: `/v3/projects/4c315743955d445bb723d05751ae155a/users/f05bc1aa5d48445f863e713b25748571/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.690533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c315743955d445bb723d05751ae155a, user_id=f05bc1aa5d48445f863e713b25748571, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:17.693206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:17.769744 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1325/2652] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:17 2022] PUT /identity/v3/projects/4c315743955d445bb723d05751ae155a/users/f05bc1aa5d48445f863e713b25748571/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:17.780396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbe3fb-ac3a-4430-ac55-8f32c652413a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.780680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbe3fb-ac3a-4430-ac55-8f32c652413a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.780984 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbe3fb-ac3a-4430-ac55-8f32c652413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.811938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6ddbe3fb-ac3a-4430-ac55-8f32c652413a None None] MFA Rules not processed for user `f05bc1aa5d48445f863e713b25748571`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:17.813172 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6ddbe3fb-ac3a-4430-ac55-8f32c652413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:17.886908 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1328/2653] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:17.892195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:17.961424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539ac9a-5f0f-4000-a465-e142f23b66b2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:17.966277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:17.966891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:17.967232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:17.967516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:17.968153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:17.970827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:18.091372 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1326/2654] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:17 2022] POST /identity/v3/projects => generated 367 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:18.095758 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbe3fb-ac3a-4430-ac55-8f32c652413a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:18.107125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbe3fb-ac3a-4430-ac55-8f32c652413a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:18.158707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:18.168285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:18.171145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:18.171145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:18.171145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:18.171145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:18.172053 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:18.291601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1329/2655] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:18 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:18.296079 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:18.307593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a05b5-ebb4-4106-bf26-e2c1b844e721 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:18.312814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:18.313964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:18.314265 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:18.314530 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:18.315337 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:18.320022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:18.334774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1327/2656] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:18.337009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:18.347805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69c4b3-c946-49d8-9785-8f06c109954c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:18.352154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:18.352888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:18.353170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:18.353882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] PATH_INFO: `/v3/projects/3e339428e0f449368bc430cef07fa951/users/6037c55656e4423897676657232e335d/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:18.371353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e339428e0f449368bc430cef07fa951, user_id=6037c55656e4423897676657232e335d, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:18.374128 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:18.452093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1330/2657] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:18 2022] PUT /identity/v3/projects/3e339428e0f449368bc430cef07fa951/users/6037c55656e4423897676657232e335d/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:18.456575 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:18.467101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214142b-f1ba-482c-b18e-d169b3a7671c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:18.545767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef371f7-11ea-45c0-8d7c-d1e2c80e3beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:18.546418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef371f7-11ea-45c0-8d7c-d1e2c80e3beb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:18.546710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef371f7-11ea-45c0-8d7c-d1e2c80e3beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:18.546976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef371f7-11ea-45c0-8d7c-d1e2c80e3beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:18.547622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef371f7-11ea-45c0-8d7c-d1e2c80e3beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:18.549627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef371f7-11ea-45c0-8d7c-d1e2c80e3beb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:18.563063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1328/2658] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:18.565284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:18.575745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe178bc-456d-4a10-ac3f-ccfdd0f20a74 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:18.580839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:18.581497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:18.582477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:18.582769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] PATH_INFO: `/v3/projects/3e339428e0f449368bc430cef07fa951/users/6037c55656e4423897676657232e335d/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:18.584804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e339428e0f449368bc430cef07fa951, user_id=6037c55656e4423897676657232e335d, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:18.587506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:18.598968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1331/2659] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:18 2022] PUT /identity/v3/projects/3e339428e0f449368bc430cef07fa951/users/6037c55656e4423897676657232e335d/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:18.611742 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41630b-2410-4cbf-8ec6-ba44fb711461 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:18.612041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41630b-2410-4cbf-8ec6-ba44fb711461 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:18.612325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41630b-2410-4cbf-8ec6-ba44fb711461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:18.643049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ad41630b-2410-4cbf-8ec6-ba44fb711461 None None] MFA Rules not processed for user `6037c55656e4423897676657232e335d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:18.644396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ad41630b-2410-4cbf-8ec6-ba44fb711461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:18.676193 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:18.716983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1329/2660] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:22:18.732216 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:18.734045 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:18.826483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298de9e-d8c4-4b69-977a-7e343e6255a8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:18.831601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:18.832077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:18.832361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:18.832637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:18.842259 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:18.843745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:18.854574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1332/2661] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:21.820615 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41630b-2410-4cbf-8ec6-ba44fb711461 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:21.908022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41630b-2410-4cbf-8ec6-ba44fb711461 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:21.913184 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:21.913872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:21.914160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:21.914438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] PATH_INFO: `/v3/users/f05bc1aa5d48445f863e713b25748571` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:21.916242 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05bc1aa5d48445f863e713b25748571)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:21.918247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:22.145539 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05bc1aa5d48445f863e713b25748571'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:22.148245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05bc1aa5d48445f863e713b25748571'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:22.160704 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05bc1aa5d48445f863e713b25748571'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:22.210611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:22.212201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:22.228232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05bc1aa5d48445f863e713b25748571'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:22.232281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1fe5dae9-d632-4cfa-8844-aa660975b8b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05bc1aa5d48445f863e713b25748571'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:22.242931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1330/2662] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:21 2022] DELETE /identity/v3/users/f05bc1aa5d48445f863e713b25748571 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:22.247469 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:22.247828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:22.248175 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:22.267670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:22.269046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:22.270367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002bc67-a84a-4d4a-a252-ed7229c07fdc service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:22.314857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:22.328974 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:22.329629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:22.329914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:22.330185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:22.331257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:22.332161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:22.361800 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1331/2663] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:22:22.370321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:22.371747 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:22.389394 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6cec7-9d40-4a84-8411-a3236b99c90f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:22.393232 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:22.394077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:22.394204 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:22.394503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:22.405448 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:22.406940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:22.413921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:22.415493 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:22.418030 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1332/2664] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:22.426852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1333/2665] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:22:22.948099 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:22.950061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:22.976198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e739383-4e93-49ae-a959-d55d4998f080 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:22.981760 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:22.982599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:22.982957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:22.983315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] PATH_INFO: `/v3/projects/4c315743955d445bb723d05751ae155a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:22.984631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c315743955d445bb723d05751ae155a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:22.986655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:23.095531 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c315743955d445bb723d05751ae155a', 'request_id': 'req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.101166 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c315743955d445bb723d05751ae155a', 'request_id': 'req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.107666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c315743955d445bb723d05751ae155a', 'request_id': 'req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.183102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] The token cache is being invalidate because project 4c315743955d445bb723d05751ae155a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:23.183401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.184190 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.185902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1333/2666] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:22 2022] DELETE /identity/v3/projects/4c315743955d445bb723d05751ae155a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:23.189894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:23.201200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da2c94-0ce5-4046-9ea1-2da83ef22b77 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:23.262018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:23.262694 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:23.262881 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:23.263157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] PATH_INFO: `/v3/users/6037c55656e4423897676657232e335d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:23.264279 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6037c55656e4423897676657232e335d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:23.266250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:23.442007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6037c55656e4423897676657232e335d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.443194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6037c55656e4423897676657232e335d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.517316 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6037c55656e4423897676657232e335d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.518788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6037c55656e4423897676657232e335d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.532034 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3d819de1-8304-4c8c-82b6-87682ec22629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6037c55656e4423897676657232e335d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:23.550615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1334/2667] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:23 2022] DELETE /identity/v3/users/6037c55656e4423897676657232e335d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:24.251297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:24.253315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:24.268484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c471d99-bfcf-4b06-a5c0-e7f5ae662115 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:24.343875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:24.344518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:24.344798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:24.345071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] PATH_INFO: `/v3/projects/3e339428e0f449368bc430cef07fa951` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:24.346303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e339428e0f449368bc430cef07fa951)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:24.348224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:24.496879 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e339428e0f449368bc430cef07fa951', 'request_id': 'req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:24.502740 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e339428e0f449368bc430cef07fa951', 'request_id': 'req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:24.504524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:24.510170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e339428e0f449368bc430cef07fa951', 'request_id': 'req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:24.610887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] The token cache is being invalidate because project 3e339428e0f449368bc430cef07fa951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:24.611417 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:24.612088 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4e0a379c-efb5-4ba4-94e2-add6acee4ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:24.613733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1334/2668] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:24 2022] DELETE /identity/v3/projects/3e339428e0f449368bc430cef07fa951 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:24.618775 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03f63-3162-48f5-9326-763a821b52ac None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:24.619263 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03f63-3162-48f5-9326-763a821b52ac None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:24.619361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03f63-3162-48f5-9326-763a821b52ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:24.640935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-aeb03f63-3162-48f5-9326-763a821b52ac None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:24.642215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aeb03f63-3162-48f5-9326-763a821b52ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:24.649844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:24.709465 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1335/2669] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:24.716945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baab2bc-8059-4541-8a72-9833ada679ac None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:24.717235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baab2bc-8059-4541-8a72-9833ada679ac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:24.717512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baab2bc-8059-4541-8a72-9833ada679ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:24.741904 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8baab2bc-8059-4541-8a72-9833ada679ac None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:24.743325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8baab2bc-8059-4541-8a72-9833ada679ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:24.752951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1335/2670] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:24.758751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03f63-3162-48f5-9326-763a821b52ac None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:24.770572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03f63-3162-48f5-9326-763a821b52ac None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:24.777978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:24.778551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:24.778840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:24.779115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:24.779787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:22:24.780088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:24.781731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:24.795430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1336/2671] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:22:24.798808 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baab2bc-8059-4541-8a72-9833ada679ac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:24.809661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baab2bc-8059-4541-8a72-9833ada679ac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:24.814555 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:24.815258 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:24.815535 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:24.815809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:24.816435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:24.819452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:24.921807 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1336/2672] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:24 2022] POST /identity/v3/projects => generated 401 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:24.926278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:24.937610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056db66-db6d-4343-be5a-778eca72aab8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.007598 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.008267 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.008551 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.008826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.009471 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.011491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:25.104196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1337/2673] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:24 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:25.108584 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:25.120207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a67fae-9963-40b0-a33f-5a1b46627351 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.125382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.130859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.130859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.130859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.130859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.130859 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:25.142981 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1337/2674] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:25.148721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:25.164910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34504c-dc2f-4dee-8964-d762f9c64da0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.170762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be880ee4-0979-467c-974e-c2dbb4250513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.171729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880ee4-0979-467c-974e-c2dbb4250513 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.172893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880ee4-0979-467c-974e-c2dbb4250513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.173278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880ee4-0979-467c-974e-c2dbb4250513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.174170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be880ee4-0979-467c-974e-c2dbb4250513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.175911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be880ee4-0979-467c-974e-c2dbb4250513 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:25.188021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1338/2675] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:25.192224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:25.203890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2f0e-4e82-459f-924e-765dce7ef60e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.209217 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.209897 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.210180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.210459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] PATH_INFO: `/v3/projects/9b122e9f5b03490b842e2a491bf6c126/users/60a32042bb724d79ac1d46ce7a848dc7/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.233756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b122e9f5b03490b842e2a491bf6c126, user_id=60a32042bb724d79ac1d46ce7a848dc7, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.236423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:25.314480 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1338/2676] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:25 2022] PUT /identity/v3/projects/9b122e9f5b03490b842e2a491bf6c126/users/60a32042bb724d79ac1d46ce7a848dc7/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:25.333514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651b9bb-fa64-4ce7-adf1-206bf86009e2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.333514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651b9bb-fa64-4ce7-adf1-206bf86009e2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.333514 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651b9bb-fa64-4ce7-adf1-206bf86009e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.357780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b651b9bb-fa64-4ce7-adf1-206bf86009e2 None None] MFA Rules not processed for user `60a32042bb724d79ac1d46ce7a848dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:25.359014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b651b9bb-fa64-4ce7-adf1-206bf86009e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:25.428486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1339/2677] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:25 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:25.433023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:25.504571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a07b3e-8f6e-4f52-95c5-df97d22d5442 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.509815 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.511136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.511136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.511376 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.512051 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.514199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:25.651905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651b9bb-fa64-4ce7-adf1-206bf86009e2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:25.653558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1339/2678] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:25 2022] POST /identity/v3/projects => generated 401 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:22:25.665597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651b9bb-fa64-4ce7-adf1-206bf86009e2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.669869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:25.733037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.733710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.734003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.734283 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.734933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.736853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:25.823116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1340/2679] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:25 2022] POST /identity/v3/users => generated 307 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:22:25.827106 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:25.838945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b761c-b43c-4ac7-a02b-203a36910cfa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.844299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.844918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.845195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.845471 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.846152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.848606 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:25.861105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1340/2680] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:25.865372 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:25.875606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30f76-7ca6-4d31-9161-a9f3a8835e1b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:25.881036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:25.887126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:25.887126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:25.887126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] PATH_INFO: `/v3/projects/2900bdff3a364c7dbfd1c1165bd7f6a5/users/66ba22248b54409aafbb801ab668b2e6/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:25.897813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2900bdff3a364c7dbfd1c1165bd7f6a5, user_id=66ba22248b54409aafbb801ab668b2e6, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:25.900655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:26.012232 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1341/2681] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:25 2022] PUT /identity/v3/projects/2900bdff3a364c7dbfd1c1165bd7f6a5/users/66ba22248b54409aafbb801ab668b2e6/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:26.016402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:26.040231 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037068b-e452-4f1e-bc46-25d26b0e25f5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:26.115090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:26.125101 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea5269-0f40-44e9-adc3-f04111729202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:26.126687 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea5269-0f40-44e9-adc3-f04111729202 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:26.126687 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea5269-0f40-44e9-adc3-f04111729202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:26.137165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea5269-0f40-44e9-adc3-f04111729202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:26.137165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea5269-0f40-44e9-adc3-f04111729202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:26.137165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea5269-0f40-44e9-adc3-f04111729202 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:26.144395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1341/2682] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:26.146287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:26.165368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f707-2358-4870-ac5a-ee2572f8f398 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:26.170439 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:26.171116 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:26.171409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:26.171676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] PATH_INFO: `/v3/projects/2900bdff3a364c7dbfd1c1165bd7f6a5/users/66ba22248b54409aafbb801ab668b2e6/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:26.180857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2900bdff3a364c7dbfd1c1165bd7f6a5, user_id=66ba22248b54409aafbb801ab668b2e6, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:26.183506 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:26.208963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1342/2683] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:26 2022] PUT /identity/v3/projects/2900bdff3a364c7dbfd1c1165bd7f6a5/users/66ba22248b54409aafbb801ab668b2e6/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:26.211147 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f4152-034a-4972-93fd-8b063806446d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:26.212122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f4152-034a-4972-93fd-8b063806446d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:26.212122 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f4152-034a-4972-93fd-8b063806446d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:26.243566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fd1f4152-034a-4972-93fd-8b063806446d None None] MFA Rules not processed for user `66ba22248b54409aafbb801ab668b2e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:26.244776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd1f4152-034a-4972-93fd-8b063806446d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:26.323791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1342/2684] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:26 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:22:26.336889 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:26.338952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:26.434192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be20378-f43a-49bc-a602-1651de1c5a40 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:26.438406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:26.439099 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:26.439380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:26.439701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:26.450271 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:26.451734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:26.475159 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1343/2685] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:26 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:26.717952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f4152-034a-4972-93fd-8b063806446d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:26.732793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f4152-034a-4972-93fd-8b063806446d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:26.741419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:26.741419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:26.741419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:26.741419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:26.803451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:26.804948 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:26.814511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1343/2686] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:26 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:30.738890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:30.741591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:30.753055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde51ea-ea29-4aac-9b04-65fa7419a52e service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:30.757338 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:30.758268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:30.758555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:30.758832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:30.760644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:30.826824 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:30.828141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:30.834079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:30.844388 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1344/2687] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:31.165666 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:31.167364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:31.248142 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563792d8-6a47-4afd-8ae4-5aee2500c115 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:31.253172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:31.253870 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:31.254285 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:31.254544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] PATH_INFO: `/v3/users/7796ec614f2545c29c2739d3dfd889f4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:31.256801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7796ec614f2545c29c2739d3dfd889f4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:31.258301 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:31.280682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:31.459683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7796ec614f2545c29c2739d3dfd889f4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:31.460268 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7796ec614f2545c29c2739d3dfd889f4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:31.477087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7796ec614f2545c29c2739d3dfd889f4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:31.548060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7796ec614f2545c29c2739d3dfd889f4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:31.548693 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7796ec614f2545c29c2739d3dfd889f4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:31.562083 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1344/2688] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:31 2022] DELETE /identity/v3/users/7796ec614f2545c29c2739d3dfd889f4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:32.046737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:32.064800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d46acf-f894-4edc-ba91-f79e2e69748b service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:32.128454 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:32.129082 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:32.129359 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:32.129672 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] PATH_INFO: `/v3/projects/770dbd9183804e7fb0cbb99af75b45f2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:32.134604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=770dbd9183804e7fb0cbb99af75b45f2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:32.134604 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:32.246655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770dbd9183804e7fb0cbb99af75b45f2', 'request_id': 'req-b127e11e-ab0e-4f21-ae86-396367b1f887'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.247776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:32.249241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:32.263012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bea37-c1b8-4b79-995c-7a21225b0d4d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:32.318486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:32.319052 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:32.319336 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:32.319612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] PATH_INFO: `/v3/users/60a32042bb724d79ac1d46ce7a848dc7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:32.320786 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a32042bb724d79ac1d46ce7a848dc7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:32.322758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:32.343087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770dbd9183804e7fb0cbb99af75b45f2', 'request_id': 'req-b127e11e-ab0e-4f21-ae86-396367b1f887'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.343087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770dbd9183804e7fb0cbb99af75b45f2', 'request_id': 'req-b127e11e-ab0e-4f21-ae86-396367b1f887'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.355591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] The token cache is being invalidate because project 770dbd9183804e7fb0cbb99af75b45f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:32.355878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.356520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.358095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1345/2689] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:32 2022] DELETE /identity/v3/projects/770dbd9183804e7fb0cbb99af75b45f2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:32.362091 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:32.363374 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:32.375937 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127e11e-ab0e-4f21-ae86-396367b1f887 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:32.381621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:32.382243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:32.382520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:32.382791 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] PATH_INFO: `/v3/users/62804425888c4f3d83991e3e3b678d30` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:32.383919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62804425888c4f3d83991e3e3b678d30)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:32.385844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:32.545211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a32042bb724d79ac1d46ce7a848dc7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.545983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a32042bb724d79ac1d46ce7a848dc7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.558475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a32042bb724d79ac1d46ce7a848dc7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.626131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62804425888c4f3d83991e3e3b678d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.626764 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62804425888c4f3d83991e3e3b678d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.650635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a32042bb724d79ac1d46ce7a848dc7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.651274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c5c4ed46-a1bf-4e47-a295-7af70c1c6969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a32042bb724d79ac1d46ce7a848dc7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.665029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1345/2690] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:32 2022] DELETE /identity/v3/users/60a32042bb724d79ac1d46ce7a848dc7 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:32.669348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:32.669669 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:32.669952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:32.751855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62804425888c4f3d83991e3e3b678d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.752535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62804425888c4f3d83991e3e3b678d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.766463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62804425888c4f3d83991e3e3b678d30'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:32.780169 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1346/2691] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:32 2022] DELETE /identity/v3/users/62804425888c4f3d83991e3e3b678d30 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:32.815706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:32.816934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:32.827424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:32.879724 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1346/2692] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:22:32.889282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:32.890771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:32.896874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:32.950779 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64b9f3-833e-4219-bd3e-e80d70e6e400 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:32.954351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:32.954948 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:32.955225 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:32.955495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:32.964610 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:32.966059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:32.975782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1347/2693] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:33.041796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:33.043511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:33.059852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48123267-c20a-4aa0-8d9f-325c5d34a004 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:33.065270 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:33.066036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:33.066543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:33.066947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] PATH_INFO: `/v3/projects/e490f07e81604d68a8aa6ccc2b3ee34d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:33.069495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e490f07e81604d68a8aa6ccc2b3ee34d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:33.072382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:33.202367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e490f07e81604d68a8aa6ccc2b3ee34d', 'request_id': 'req-a38bf94c-7e4e-423c-92be-9546c62f44db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.215668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e490f07e81604d68a8aa6ccc2b3ee34d', 'request_id': 'req-a38bf94c-7e4e-423c-92be-9546c62f44db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.216918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e490f07e81604d68a8aa6ccc2b3ee34d', 'request_id': 'req-a38bf94c-7e4e-423c-92be-9546c62f44db'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.296853 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] The token cache is being invalidate because project e490f07e81604d68a8aa6ccc2b3ee34d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:33.297478 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.298011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.299486 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1347/2694] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:33 2022] DELETE /identity/v3/projects/e490f07e81604d68a8aa6ccc2b3ee34d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:33.407649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:33.409524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:33.441386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aa9a9-b655-45e9-a3bc-15c04fd82a71 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:33.510153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:33.510810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:33.511089 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:33.511369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] PATH_INFO: `/v3/projects/9b122e9f5b03490b842e2a491bf6c126` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:33.512577 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b122e9f5b03490b842e2a491bf6c126)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:33.514852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:33.640449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b122e9f5b03490b842e2a491bf6c126', 'request_id': 'req-b103c9c7-46ba-48f4-a358-07e6c12eb629'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.742340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b122e9f5b03490b842e2a491bf6c126', 'request_id': 'req-b103c9c7-46ba-48f4-a358-07e6c12eb629'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.748370 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b122e9f5b03490b842e2a491bf6c126', 'request_id': 'req-b103c9c7-46ba-48f4-a358-07e6c12eb629'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.760044 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] The token cache is being invalidate because project 9b122e9f5b03490b842e2a491bf6c126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:33.760342 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.760994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.762516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1348/2695] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:33 2022] DELETE /identity/v3/projects/9b122e9f5b03490b842e2a491bf6c126 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:33.766577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:33.767983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:33.781580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf94c-7e4e-423c-92be-9546c62f44db admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:33.837863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:33.838494 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:33.838774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:33.839049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] PATH_INFO: `/v3/users/66ba22248b54409aafbb801ab668b2e6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:33.840163 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ba22248b54409aafbb801ab668b2e6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:33.842127 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:33.985796 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:33.991638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ba22248b54409aafbb801ab668b2e6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:33.992428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ba22248b54409aafbb801ab668b2e6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.004211 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ba22248b54409aafbb801ab668b2e6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.085195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ba22248b54409aafbb801ab668b2e6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.085837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03f5fc2-287b-4b62-aa86-0840766205d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ba22248b54409aafbb801ab668b2e6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.098219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1348/2696] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:33 2022] DELETE /identity/v3/users/66ba22248b54409aafbb801ab668b2e6 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:34.465422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:34.465422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:34.479354 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103c9c7-46ba-48f4-a358-07e6c12eb629 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:34.517931 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:34.543103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:34.543669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:34.543952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:34.544250 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] PATH_INFO: `/v3/projects/2900bdff3a364c7dbfd1c1165bd7f6a5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:34.545382 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2900bdff3a364c7dbfd1c1165bd7f6a5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:34.547307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:34.711576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2900bdff3a364c7dbfd1c1165bd7f6a5', 'request_id': 'req-bc19834a-c545-433a-8959-48319f5fb760'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.776665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2900bdff3a364c7dbfd1c1165bd7f6a5', 'request_id': 'req-bc19834a-c545-433a-8959-48319f5fb760'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.782767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2900bdff3a364c7dbfd1c1165bd7f6a5', 'request_id': 'req-bc19834a-c545-433a-8959-48319f5fb760'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.794951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] The token cache is being invalidate because project 2900bdff3a364c7dbfd1c1165bd7f6a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:34.795185 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.795819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bc19834a-c545-433a-8959-48319f5fb760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:34.797300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1349/2697] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:34 2022] DELETE /identity/v3/projects/2900bdff3a364c7dbfd1c1165bd7f6a5 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:34.803277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9af62-f40d-433b-b14c-02c86f14d026 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:34.803889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9af62-f40d-433b-b14c-02c86f14d026 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:34.803889 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9af62-f40d-433b-b14c-02c86f14d026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:34.823827 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-02e9af62-f40d-433b-b14c-02c86f14d026 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:34.825094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-02e9af62-f40d-433b-b14c-02c86f14d026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:34.884326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1349/2698] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:34.892293 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d36fb-e5a9-440b-bc2d-f3e2e241121c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:34.892584 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d36fb-e5a9-440b-bc2d-f3e2e241121c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:34.892863 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d36fb-e5a9-440b-bc2d-f3e2e241121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:34.913973 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-db2d36fb-e5a9-440b-bc2d-f3e2e241121c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:34.915201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-db2d36fb-e5a9-440b-bc2d-f3e2e241121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:34.923097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1350/2699] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:34.928055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9af62-f40d-433b-b14c-02c86f14d026 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:34.937708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9af62-f40d-433b-b14c-02c86f14d026 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:34.942238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:34.942836 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:34.943114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:34.943388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:34.944026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:22:34.944348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:34.945997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:34.957134 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1350/2700] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:22:34.961444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d36fb-e5a9-440b-bc2d-f3e2e241121c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:34.970562 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d36fb-e5a9-440b-bc2d-f3e2e241121c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:34.975765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:34.976350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:34.976629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:34.976898 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:34.977515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:34.979415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.064050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1351/2701] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:34 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:35.067477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.077286 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab4656-5a3e-49b6-a913-d6deb894003a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.132412 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.133005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.133283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.133554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.134224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.136172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.221764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1351/2702] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:35 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:35.226709 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.236772 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629119f-c583-4476-9f69-9a078ddcf7e0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.241410 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.242028 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.242307 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.242578 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.243213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.244917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.254603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1352/2703] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:35.258672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.268453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc28f26-c193-49f1-b44f-f3d9fb4285c4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.273976 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cc19c-120f-4e68-8261-79f6ba146e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.274580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cc19c-120f-4e68-8261-79f6ba146e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.274862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cc19c-120f-4e68-8261-79f6ba146e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.275138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cc19c-120f-4e68-8261-79f6ba146e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.275777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cc19c-120f-4e68-8261-79f6ba146e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.277695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cc19c-120f-4e68-8261-79f6ba146e76 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.288178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1352/2704] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:35.292326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.302603 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff6c28-a60c-44f2-80b2-5709726ef99a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.307209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.307851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.308132 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.308407 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] PATH_INFO: `/v3/projects/38efe268d4b94e6cbcb683ea40440961/users/a2bbd008271e4296be2b23294786346e/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.323451 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38efe268d4b94e6cbcb683ea40440961, user_id=a2bbd008271e4296be2b23294786346e, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.326054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.398050 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1353/2705] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:35 2022] PUT /identity/v3/projects/38efe268d4b94e6cbcb683ea40440961/users/a2bbd008271e4296be2b23294786346e/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:35.410297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d460-1116-4376-9517-8de08a20b04d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.410596 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d460-1116-4376-9517-8de08a20b04d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.410874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d460-1116-4376-9517-8de08a20b04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.438999 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dae4d460-1116-4376-9517-8de08a20b04d None None] MFA Rules not processed for user `a2bbd008271e4296be2b23294786346e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:35.440200 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dae4d460-1116-4376-9517-8de08a20b04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:35.452389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:35.506111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1353/2706] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:35.509870 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.576597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9073a-17b8-4be2-a116-955becfd46a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.581412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.582055 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.589438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.589438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.589438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.589438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.591877 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:35.704733 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1354/2707] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:35 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:35.709294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d460-1116-4376-9517-8de08a20b04d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.720373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d460-1116-4376-9517-8de08a20b04d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.780017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.780629 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.780912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.781188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.781854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.783771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.866882 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1354/2708] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:35 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:35.871061 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.882922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc35f3a-e10d-477c-b859-41448954c9d4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.887555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.888192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.888470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.888763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.889386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.891005 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.901066 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1355/2709] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:35.905273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.915378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58890bd-f148-4d9c-bbf8-b947bbbedaab admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.920109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b7a67-46f6-45c3-8c38-13ffd38a4501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.920814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7a67-46f6-45c3-8c38-13ffd38a4501 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.921095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7a67-46f6-45c3-8c38-13ffd38a4501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.921392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7a67-46f6-45c3-8c38-13ffd38a4501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.922081 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b7a67-46f6-45c3-8c38-13ffd38a4501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.923703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b7a67-46f6-45c3-8c38-13ffd38a4501 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:35.934569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1355/2710] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:35.938895 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:35.949526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251dcbe-3b57-453e-a5ca-7633f992ea20 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:35.954084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:35.955446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:35.955446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:35.955446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] PATH_INFO: `/v3/projects/4bcbde25822640daa9719e9f58ac5fb9/users/eee861c06a494a3b8b50058f6f9476c9/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:35.969555 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bcbde25822640daa9719e9f58ac5fb9, user_id=eee861c06a494a3b8b50058f6f9476c9, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:35.972192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:36.050865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1356/2711] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:35 2022] PUT /identity/v3/projects/4bcbde25822640daa9719e9f58ac5fb9/users/eee861c06a494a3b8b50058f6f9476c9/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:36.062267 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee22a84-b521-4d38-8b63-fc6230f55575 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.062557 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee22a84-b521-4d38-8b63-fc6230f55575 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.062834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee22a84-b521-4d38-8b63-fc6230f55575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.090903 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aee22a84-b521-4d38-8b63-fc6230f55575 None None] MFA Rules not processed for user `eee861c06a494a3b8b50058f6f9476c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:36.092086 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aee22a84-b521-4d38-8b63-fc6230f55575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:36.169046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1356/2712] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:36.173699 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:36.236793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bae9c-d387-4dc9-8a9e-adcb8d33a33c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:36.242258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:36.243072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.243356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.243628 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.244285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:36.246236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:36.336706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1357/2713] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:36 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:22:36.340810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee22a84-b521-4d38-8b63-fc6230f55575 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:36.352628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee22a84-b521-4d38-8b63-fc6230f55575 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:36.377893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:36.410600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:36.411764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.411764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.411764 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.412388 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:36.414732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:36.495403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1357/2714] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:36 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:36.499452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:36.511313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c43b-52b4-4f12-9c3d-e411cb5dbdaf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:36.516143 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:36.516741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.517021 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.517379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.518432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:36.520020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:36.531670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1358/2715] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:36.536259 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:36.546635 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74929dc-bb36-40b8-89d2-2f034a404da6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:36.551462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:36.552109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.552389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.552663 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] PATH_INFO: `/v3/projects/f1068194346a4479a7d598175364d18a/users/cd6569d8ae574c1c9d6700f01346f150/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.567558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1068194346a4479a7d598175364d18a, user_id=cd6569d8ae574c1c9d6700f01346f150, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:36.570880 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:36.639673 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1358/2716] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:36 2022] PUT /identity/v3/projects/f1068194346a4479a7d598175364d18a/users/cd6569d8ae574c1c9d6700f01346f150/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:36.643601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:36.654110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b6bd0-dffd-4aaa-9449-0f0cf708fd69 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:36.709778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751df08e-859e-4fa8-8eb1-a51e30e5ca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:36.710582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751df08e-859e-4fa8-8eb1-a51e30e5ca21 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.710869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751df08e-859e-4fa8-8eb1-a51e30e5ca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.711145 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751df08e-859e-4fa8-8eb1-a51e30e5ca21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.711821 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751df08e-859e-4fa8-8eb1-a51e30e5ca21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:36.713405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751df08e-859e-4fa8-8eb1-a51e30e5ca21 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:36.722880 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1359/2717] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:36.727308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:36.737501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8f409-bbd8-466a-b260-14adcfeb45cc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:36.742712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:36.743358 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.743638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.743909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] PATH_INFO: `/v3/projects/f1068194346a4479a7d598175364d18a/users/cd6569d8ae574c1c9d6700f01346f150/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.745847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1068194346a4479a7d598175364d18a, user_id=cd6569d8ae574c1c9d6700f01346f150, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:36.748488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:36.758628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1359/2718] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:36 2022] PUT /identity/v3/projects/f1068194346a4479a7d598175364d18a/users/cd6569d8ae574c1c9d6700f01346f150/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:36.768767 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4a3c-c303-4db2-a89f-31d653375d53 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.769054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4a3c-c303-4db2-a89f-31d653375d53 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.769327 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a4a3c-c303-4db2-a89f-31d653375d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.798001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-163a4a3c-c303-4db2-a89f-31d653375d53 None None] MFA Rules not processed for user `cd6569d8ae574c1c9d6700f01346f150`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:36.799192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-163a4a3c-c303-4db2-a89f-31d653375d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:36.808077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:36.867076 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1360/2719] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:36.878783 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:36.880501 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:36.938255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea8a30-7676-46e2-b30b-26dc53bc06d1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:36.941830 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78956d-aa25-46af-83ef-9457b2eea93e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:36.942430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78956d-aa25-46af-83ef-9457b2eea93e service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:36.942712 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78956d-aa25-46af-83ef-9457b2eea93e service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:36.942993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78956d-aa25-46af-83ef-9457b2eea93e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:36.998362 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78956d-aa25-46af-83ef-9457b2eea93e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:36.999788 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78956d-aa25-46af-83ef-9457b2eea93e service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:37.008895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1360/2720] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:39.183016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a4a3c-c303-4db2-a89f-31d653375d53 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:39.192594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a4a3c-c303-4db2-a89f-31d653375d53 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:39.196443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:39.197052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:39.197369 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:39.197670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:39.253474 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:39.254922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:39.264286 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1361/2721] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:39.414378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ed5e-1a49-495e-b87d-c2a0f37f74a6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:39.414919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ed5e-1a49-495e-b87d-c2a0f37f74a6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:39.414919 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ed5e-1a49-495e-b87d-c2a0f37f74a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:39.434178 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a159ed5e-1a49-495e-b87d-c2a0f37f74a6 None None] MFA Rules not processed for user `b694f1163ef54b82b86e06bf825c57e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:39.435383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a159ed5e-1a49-495e-b87d-c2a0f37f74a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:39.458898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:39.491899 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1361/2722] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:40.697867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:40.707115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9c508-ff4f-43c5-8f0b-64dcbf50e33c service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:40.711205 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55297e-8458-4eea-a5d7-ced80d4b9f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:40.711811 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55297e-8458-4eea-a5d7-ced80d4b9f5e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:40.712118 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55297e-8458-4eea-a5d7-ced80d4b9f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:40.712400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55297e-8458-4eea-a5d7-ced80d4b9f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:40.721939 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55297e-8458-4eea-a5d7-ced80d4b9f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:40.723350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55297e-8458-4eea-a5d7-ced80d4b9f5e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:40.733750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1362/2723] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:45.543638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159ed5e-1a49-495e-b87d-c2a0f37f74a6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:45.606102 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159ed5e-1a49-495e-b87d-c2a0f37f74a6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:45.610630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:45.611276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:45.611583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:45.611854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] PATH_INFO: `/v3/users/a2bbd008271e4296be2b23294786346e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:45.612979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2bbd008271e4296be2b23294786346e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:45.614920 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:45.750128 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2bbd008271e4296be2b23294786346e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:45.750784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2bbd008271e4296be2b23294786346e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:45.762944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2bbd008271e4296be2b23294786346e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:45.842521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2bbd008271e4296be2b23294786346e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:45.843139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2bbd008271e4296be2b23294786346e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:45.857804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1362/2724] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:45 2022] DELETE /identity/v3/users/a2bbd008271e4296be2b23294786346e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:45.860652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff219-0368-468b-97ba-dada4883279a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:45.861131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff219-0368-468b-97ba-dada4883279a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:45.861252 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff219-0368-468b-97ba-dada4883279a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:45.880324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f83ff219-0368-468b-97ba-dada4883279a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:45.881552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f83ff219-0368-468b-97ba-dada4883279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:45.941361 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1363/2725] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:45.951300 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:45.952716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:46.014468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf79c1b-59a6-4ea5-a9d8-1c82f3f53e38 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:46.019245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:46.019857 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:46.020134 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:46.020403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:46.029111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:46.031586 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:46.033144 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:46.044036 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1363/2726] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:46.446278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ff219-0368-468b-97ba-dada4883279a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:46.448820 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f83ff219-0368-468b-97ba-dada4883279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:46.460995 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ff219-0368-468b-97ba-dada4883279a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:46.467633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:46.468653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:46.468919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:46.470181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] PATH_INFO: `/v3/projects/38efe268d4b94e6cbcb683ea40440961` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:46.471994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38efe268d4b94e6cbcb683ea40440961)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:46.474100 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:46.571967 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:46.587165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38efe268d4b94e6cbcb683ea40440961', 'request_id': 'req-a76f3490-c36d-4661-bbf5-e80f291bed8d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:46.654700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38efe268d4b94e6cbcb683ea40440961', 'request_id': 'req-a76f3490-c36d-4661-bbf5-e80f291bed8d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:46.660655 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38efe268d4b94e6cbcb683ea40440961', 'request_id': 'req-a76f3490-c36d-4661-bbf5-e80f291bed8d'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:46.672893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] The token cache is being invalidate because project 38efe268d4b94e6cbcb683ea40440961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:46.673183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:46.673852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:46.675350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1364/2727] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:46 2022] DELETE /identity/v3/projects/38efe268d4b94e6cbcb683ea40440961 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:46.679410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:46.689351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38217f92-1260-4da6-ab8a-e9f5f748db1a service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:46.743424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:46.744059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:46.744348 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:46.744668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] PATH_INFO: `/v3/users/eee861c06a494a3b8b50058f6f9476c9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:46.745823 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee861c06a494a3b8b50058f6f9476c9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:46.747725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:46.932016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee861c06a494a3b8b50058f6f9476c9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:46.932610 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee861c06a494a3b8b50058f6f9476c9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:46.944207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee861c06a494a3b8b50058f6f9476c9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.014229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee861c06a494a3b8b50058f6f9476c9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.014855 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee861c06a494a3b8b50058f6f9476c9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.028346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1364/2728] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:46 2022] DELETE /identity/v3/users/eee861c06a494a3b8b50058f6f9476c9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:47.450845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:47.450845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:47.464000 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f3490-c36d-4661-bbf5-e80f291bed8d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:47.522295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:47.522922 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:47.523201 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:47.523472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] PATH_INFO: `/v3/projects/4bcbde25822640daa9719e9f58ac5fb9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:47.524576 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bcbde25822640daa9719e9f58ac5fb9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:47.526654 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:47.640115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bcbde25822640daa9719e9f58ac5fb9', 'request_id': 'req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.711884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bcbde25822640daa9719e9f58ac5fb9', 'request_id': 'req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.718866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bcbde25822640daa9719e9f58ac5fb9', 'request_id': 'req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.731042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] The token cache is being invalidate because project 4bcbde25822640daa9719e9f58ac5fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:47.731350 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.731999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:47.733572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1365/2729] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:47 2022] DELETE /identity/v3/projects/4bcbde25822640daa9719e9f58ac5fb9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:47.737657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:47.739194 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:47.752256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cde55a-790b-4bfd-8d5b-dd898f1ae56b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:47.808481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:47.809165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:47.809438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:47.809743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] PATH_INFO: `/v3/users/cd6569d8ae574c1c9d6700f01346f150` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:47.810851 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd6569d8ae574c1c9d6700f01346f150)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:47.812791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:47.828403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:48.056544 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd6569d8ae574c1c9d6700f01346f150'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.057236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd6569d8ae574c1c9d6700f01346f150'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.069927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd6569d8ae574c1c9d6700f01346f150'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.147957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd6569d8ae574c1c9d6700f01346f150'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.148569 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-47b4047c-a405-4004-b463-fad1b9411442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd6569d8ae574c1c9d6700f01346f150'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.161138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1365/2730] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:47 2022] DELETE /identity/v3/users/cd6569d8ae574c1c9d6700f01346f150 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:48.559092 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:48.560525 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:48.573258 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ca222-f7e5-451d-b381-b6fb7a4d1f04 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:48.620969 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:48.633237 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:48.633899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:48.634180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:48.634449 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] PATH_INFO: `/v3/projects/f1068194346a4479a7d598175364d18a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:48.635559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1068194346a4479a7d598175364d18a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:48.638214 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:48.742399 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1068194346a4479a7d598175364d18a', 'request_id': 'req-7eb21252-b882-41e4-956b-e45f5ba7b4eb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.822212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1068194346a4479a7d598175364d18a', 'request_id': 'req-7eb21252-b882-41e4-956b-e45f5ba7b4eb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.828990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1068194346a4479a7d598175364d18a', 'request_id': 'req-7eb21252-b882-41e4-956b-e45f5ba7b4eb'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.841151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] The token cache is being invalidate because project f1068194346a4479a7d598175364d18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:22:48.843754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.843754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7eb21252-b882-41e4-956b-e45f5ba7b4eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:48.843754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1366/2731] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:22:48 2022] DELETE /identity/v3/projects/f1068194346a4479a7d598175364d18a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:48.848572 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c9f9-622f-4046-9fdb-e0157f2fcb36 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:48.849418 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c9f9-622f-4046-9fdb-e0157f2fcb36 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:48.849719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899c9f9-622f-4046-9fdb-e0157f2fcb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:48.869488 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8899c9f9-622f-4046-9fdb-e0157f2fcb36 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:48.870718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8899c9f9-622f-4046-9fdb-e0157f2fcb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:48.929843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1366/2732] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:22:48.992484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484ed52-72a5-42ea-9ab4-44fcefd2a2e1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:48.992745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484ed52-72a5-42ea-9ab4-44fcefd2a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:48.993022 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484ed52-72a5-42ea-9ab4-44fcefd2a2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.013550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7484ed52-72a5-42ea-9ab4-44fcefd2a2e1 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:49.014782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7484ed52-72a5-42ea-9ab4-44fcefd2a2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:49.022738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1367/2733] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:49.027066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c9f9-622f-4046-9fdb-e0157f2fcb36 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.037005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899c9f9-622f-4046-9fdb-e0157f2fcb36 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.041811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.042397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.042727 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.043006 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.043641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:22:49.043941 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.045528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.056624 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1367/2734] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:22:49.061295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484ed52-72a5-42ea-9ab4-44fcefd2a2e1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.070346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484ed52-72a5-42ea-9ab4-44fcefd2a2e1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.075633 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.076155 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.076430 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.076701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.077334 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.079229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.155195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1368/2735] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:49 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:49.159403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.170093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754960-9082-4968-8099-f1c5d6a140af admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.223927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.224525 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.224832 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.225107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.225768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.227661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.297393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1368/2736] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:49 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:49.301608 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.313027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82028049-4f89-4c33-ba07-35407a35e64b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.317920 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.318526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.318829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.319084 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.319759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.321356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.331951 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1369/2737] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:49.335940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.346616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41fdc4-d257-4273-8a36-3336120874f3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.351212 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320a222-4780-414f-841e-4ec2438d3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.351802 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320a222-4780-414f-841e-4ec2438d3375 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.352084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320a222-4780-414f-841e-4ec2438d3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.352357 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320a222-4780-414f-841e-4ec2438d3375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.352992 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320a222-4780-414f-841e-4ec2438d3375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.354618 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320a222-4780-414f-841e-4ec2438d3375 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.360739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:49.366254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1369/2738] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:22:49.374652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.384661 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181748c-dbfa-4ffe-961a-a22929b78993 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.389635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.390280 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.390556 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.390826 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] PATH_INFO: `/v3/projects/c4b5b9102ec94dc1ba38ffd29163064d/users/e18b8ff797dc4d938a6ab4916da7d51b/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.406567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b5b9102ec94dc1ba38ffd29163064d, user_id=e18b8ff797dc4d938a6ab4916da7d51b, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.409287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.468275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1370/2739] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:49 2022] PUT /identity/v3/projects/c4b5b9102ec94dc1ba38ffd29163064d/users/e18b8ff797dc4d938a6ab4916da7d51b/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:49.478491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b9d00-ddcc-4491-bd19-240d4fdcd63c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.478814 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b9d00-ddcc-4491-bd19-240d4fdcd63c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.479096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b9d00-ddcc-4491-bd19-240d4fdcd63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.510446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7a8b9d00-ddcc-4491-bd19-240d4fdcd63c None None] MFA Rules not processed for user `e18b8ff797dc4d938a6ab4916da7d51b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:49.512016 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7a8b9d00-ddcc-4491-bd19-240d4fdcd63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:49.576499 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1370/2740] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:22:49.581261 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.643721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337b6c2-a544-48a6-a638-79ab31e09826 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.648311 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.648933 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.649196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.649469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.650142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.652535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.732296 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:49.742800 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1371/2741] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:49 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:49.747098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b9d00-ddcc-4491-bd19-240d4fdcd63c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.757991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b9d00-ddcc-4491-bd19-240d4fdcd63c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.815944 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.816554 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.816833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.817109 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.817767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.819717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.937332 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1371/2742] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:49 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:49.941390 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.952297 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c21669-b4c6-489a-85fb-63b4dcf57f63 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.957046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.957671 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.957952 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.958224 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.958856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.960446 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:49.975186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1372/2743] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:22:49.976504 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:49.986574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fc80e-915f-4708-b0d5-ade77be01833 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:49.991561 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84032c58-337d-412a-ab3d-61d3aa634a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:49.992165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84032c58-337d-412a-ab3d-61d3aa634a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:49.992445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84032c58-337d-412a-ab3d-61d3aa634a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:49.992743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84032c58-337d-412a-ab3d-61d3aa634a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:49.993378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84032c58-337d-412a-ab3d-61d3aa634a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:49.995045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84032c58-337d-412a-ab3d-61d3aa634a9e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.005346 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1372/2744] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:50.009643 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.020397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a764b2-a341-48f0-ac1f-b9821a07e0cd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.025138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.025819 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.026098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.026368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] PATH_INFO: `/v3/projects/69d7f506e33945ebb592c64a179a3e54/users/f7813dbbfe664bc28b3b905f6e33ee6d/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.044343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d7f506e33945ebb592c64a179a3e54, user_id=f7813dbbfe664bc28b3b905f6e33ee6d, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:50.047006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.117083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1373/2745] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:50 2022] PUT /identity/v3/projects/69d7f506e33945ebb592c64a179a3e54/users/f7813dbbfe664bc28b3b905f6e33ee6d/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:22:50.129277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24615ce8-9ccc-4a86-994c-975b25f265cf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.129602 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24615ce8-9ccc-4a86-994c-975b25f265cf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.129886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24615ce8-9ccc-4a86-994c-975b25f265cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.157852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-24615ce8-9ccc-4a86-994c-975b25f265cf None None] MFA Rules not processed for user `f7813dbbfe664bc28b3b905f6e33ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:50.159074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-24615ce8-9ccc-4a86-994c-975b25f265cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:50.216964 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:50.222089 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1373/2746] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:50.226533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.288750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ad98d-dde8-4ad7-bdef-6580c4233bbb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.294429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.295042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.295324 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.295601 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.296244 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:50.298178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.383962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1374/2747] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:22:50 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:50.389022 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24615ce8-9ccc-4a86-994c-975b25f265cf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.399456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24615ce8-9ccc-4a86-994c-975b25f265cf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.456902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.457510 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.457825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.458098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.458719 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:50.460672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.540545 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1374/2748] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:22:50 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:22:50.545220 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.555554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e3cc7-f017-4e53-acc1-1047d36856c3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.560468 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.561072 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.561353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.561668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.562310 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:50.563907 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.574502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1375/2749] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:50.578717 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.588626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864414a-1d00-47da-a399-4f1ac9e7ca6d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.594104 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.594749 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.595034 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.595313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] PATH_INFO: `/v3/projects/77dbb49b27964715835ed8a9472fd309/users/650926e295284d18978f9ff832cc70e4/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.610467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77dbb49b27964715835ed8a9472fd309, user_id=650926e295284d18978f9ff832cc70e4, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:50.613170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.679392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1375/2750] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:50 2022] PUT /identity/v3/projects/77dbb49b27964715835ed8a9472fd309/users/650926e295284d18978f9ff832cc70e4/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:50.683516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.693550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf14e5-ee78-4edb-9804-0514afbc9036 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.749360 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f9fd7-db61-4c9b-aaec-ed60dd51d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.750011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f9fd7-db61-4c9b-aaec-ed60dd51d912 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.750295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f9fd7-db61-4c9b-aaec-ed60dd51d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.750567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f9fd7-db61-4c9b-aaec-ed60dd51d912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.751203 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f9fd7-db61-4c9b-aaec-ed60dd51d912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:50.752787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f9fd7-db61-4c9b-aaec-ed60dd51d912 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.762972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1376/2751] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:22:50.767151 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.777828 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27443c2-0878-4bad-ba2d-f6f733ca562c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.782442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.783093 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.783373 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.783642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] PATH_INFO: `/v3/projects/77dbb49b27964715835ed8a9472fd309/users/650926e295284d18978f9ff832cc70e4/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.785549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77dbb49b27964715835ed8a9472fd309, user_id=650926e295284d18978f9ff832cc70e4, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:50.788189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:50.798431 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1376/2752] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:22:50 2022] PUT /identity/v3/projects/77dbb49b27964715835ed8a9472fd309/users/650926e295284d18978f9ff832cc70e4/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:50.808605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.808963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.809253 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:50.837832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] MFA Rules not processed for user `650926e295284d18978f9ff832cc70e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:50.839035 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:50.861747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:50.909046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1377/2753] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:50.920399 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:50.922095 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:50.979496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a29b8-05b3-4d95-8616-2c86ff1cee02 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:50.983414 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:50.984015 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:50.984297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:50.984571 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:51.040805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:51.042427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:51.052577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1377/2754] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:52.973660 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:52.975039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:52.987659 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62bba7-dacd-4efd-8257-70c47d3114cf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:52.991309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:52.991905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:52.992178 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:52.992448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:53.048778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:53.050233 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:53.059572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1378/2755] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:54.455157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:54.460971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:54.464503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801e80f-3d0c-4005-8af8-b6d0f27d7e66 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:54.470257 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:54.470994 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:54.471354 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:54.471702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:54.481188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:54.482672 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:54.497743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1378/2756] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:22:57.302450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:57.303875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:57.365204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e7431-b263-4fe5-a6dc-522bc449547d service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:57.368850 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f23b72-efd9-478a-883a-e6abe12a9b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:57.369450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f23b72-efd9-478a-883a-e6abe12a9b43 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:57.369769 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f23b72-efd9-478a-883a-e6abe12a9b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:57.370437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f23b72-efd9-478a-883a-e6abe12a9b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:57.372762 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d3f23b72-efd9-478a-883a-e6abe12a9b43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:57.436218 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f23b72-efd9-478a-883a-e6abe12a9b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:57.437702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f23b72-efd9-478a-883a-e6abe12a9b43 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:57.447606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1379/2757] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:22:59.033620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:59.097120 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df6a20-1a12-4f88-82bd-4271b5264204 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:59.101696 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:59.102319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:59.102600 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:59.102875 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] PATH_INFO: `/v3/users/e18b8ff797dc4d938a6ab4916da7d51b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:59.103993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18b8ff797dc4d938a6ab4916da7d51b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:59.106521 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:59.305383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18b8ff797dc4d938a6ab4916da7d51b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:59.306261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18b8ff797dc4d938a6ab4916da7d51b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:59.320249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18b8ff797dc4d938a6ab4916da7d51b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:59.411612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18b8ff797dc4d938a6ab4916da7d51b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:59.412619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18b8ff797dc4d938a6ab4916da7d51b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:22:59.429216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1379/2758] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:22:59 2022] DELETE /identity/v3/users/e18b8ff797dc4d938a6ab4916da7d51b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:22:59.434750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:59.435748 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:59.436326 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:59.453635 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:22:59.460377 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:22:59.461780 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:59.545744 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1380/2759] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:22:59.552726 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:22:59.555009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:22:59.627706 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644742e-6dc8-4810-b57f-36ba8688b0e8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:22:59.632304 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:22:59.632907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:22:59.633226 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:22:59.633503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:22:59.644621 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:22:59.646277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:22:59.657803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1380/2760] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 16:23:00.086083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:00.087103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:00.112770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f891c3-1211-4342-8558-e81027990e75 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:00.122126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:00.125227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:00.125729 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:00.126103 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] PATH_INFO: `/v3/projects/c4b5b9102ec94dc1ba38ffd29163064d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:00.127343 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b5b9102ec94dc1ba38ffd29163064d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:00.130295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:00.241346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b5b9102ec94dc1ba38ffd29163064d', 'request_id': 'req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.325597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b5b9102ec94dc1ba38ffd29163064d', 'request_id': 'req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.340523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b5b9102ec94dc1ba38ffd29163064d', 'request_id': 'req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.356498 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] The token cache is being invalidate because project c4b5b9102ec94dc1ba38ffd29163064d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:23:00.356787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.357722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.359581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1381/2761] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:23:00 2022] DELETE /identity/v3/projects/c4b5b9102ec94dc1ba38ffd29163064d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:23:00.363784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:00.374767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b3128-16f0-4ed6-bda5-32db568006a7 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:00.385046 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:00.444428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:00.445427 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:00.445743 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:00.446018 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] PATH_INFO: `/v3/users/f7813dbbfe664bc28b3b905f6e33ee6d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:00.447169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7813dbbfe664bc28b3b905f6e33ee6d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:00.449718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:00.653277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7813dbbfe664bc28b3b905f6e33ee6d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.653956 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7813dbbfe664bc28b3b905f6e33ee6d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.665445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7813dbbfe664bc28b3b905f6e33ee6d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.734322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7813dbbfe664bc28b3b905f6e33ee6d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.734947 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7813dbbfe664bc28b3b905f6e33ee6d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:00.747438 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1381/2762] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:23:00 2022] DELETE /identity/v3/users/f7813dbbfe664bc28b3b905f6e33ee6d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:23:01.153750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:01.155221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:01.170482 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1f1a-5c75-4256-b03d-c8eb8783b17f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:01.226959 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:01.227592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:01.227875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:01.228148 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] PATH_INFO: `/v3/projects/69d7f506e33945ebb592c64a179a3e54` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:01.229266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d7f506e33945ebb592c64a179a3e54)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:01.231749 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:01.352444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:01.364332 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d7f506e33945ebb592c64a179a3e54', 'request_id': 'req-1cf0d840-d370-4126-9243-3af8e3212033'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.426179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d7f506e33945ebb592c64a179a3e54', 'request_id': 'req-1cf0d840-d370-4126-9243-3af8e3212033'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.432934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d7f506e33945ebb592c64a179a3e54', 'request_id': 'req-1cf0d840-d370-4126-9243-3af8e3212033'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.445810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] The token cache is being invalidate because project 69d7f506e33945ebb592c64a179a3e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:23:01.446102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.446759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.448247 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1382/2763] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:23:01 2022] DELETE /identity/v3/projects/69d7f506e33945ebb592c64a179a3e54 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:23:01.452266 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:01.453775 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:01.466238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192061c-14f9-4d00-b635-22f7075e6762 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:01.522798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:01.523492 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:01.523778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:01.524188 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] PATH_INFO: `/v3/users/650926e295284d18978f9ff832cc70e4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:01.525311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=650926e295284d18978f9ff832cc70e4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:01.527351 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:01.705626 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:01.715463 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '650926e295284d18978f9ff832cc70e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.716150 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '650926e295284d18978f9ff832cc70e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.728271 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '650926e295284d18978f9ff832cc70e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.786274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '650926e295284d18978f9ff832cc70e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.786959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-937cfa8f-6562-4684-a6ea-2e4fb7cdff02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '650926e295284d18978f9ff832cc70e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:01.801441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1382/2764] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:23:01 2022] DELETE /identity/v3/users/650926e295284d18978f9ff832cc70e4 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:23:02.229879 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:02.231875 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:02.245274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d840-d370-4126-9243-3af8e3212033 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:02.303302 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:02.303930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:02.304215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:02.304491 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] PATH_INFO: `/v3/projects/77dbb49b27964715835ed8a9472fd309` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:02.305637 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] RBAC: Authorizing `identity:delete_project(project_id=77dbb49b27964715835ed8a9472fd309)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:02.307530 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:02.412179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77dbb49b27964715835ed8a9472fd309', 'request_id': 'req-0e7f622f-de39-45ca-b5ce-6391717df2be'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:02.502757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77dbb49b27964715835ed8a9472fd309', 'request_id': 'req-0e7f622f-de39-45ca-b5ce-6391717df2be'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:02.509273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77dbb49b27964715835ed8a9472fd309', 'request_id': 'req-0e7f622f-de39-45ca-b5ce-6391717df2be'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:02.521032 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] The token cache is being invalidate because project 77dbb49b27964715835ed8a9472fd309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:23:02.521319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:02.521993 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0e7f622f-de39-45ca-b5ce-6391717df2be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:02.523490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1383/2765] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:23:02 2022] DELETE /identity/v3/projects/77dbb49b27964715835ed8a9472fd309 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:23:02.529237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bffcb-2f46-4dc9-ba47-7ffe97a23670 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:02.529951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bffcb-2f46-4dc9-ba47-7ffe97a23670 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:02.529951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bffcb-2f46-4dc9-ba47-7ffe97a23670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:02.550214 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6a3bffcb-2f46-4dc9-ba47-7ffe97a23670 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:23:02.551468 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6a3bffcb-2f46-4dc9-ba47-7ffe97a23670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:02.612756 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1383/2766] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:23:02.620728 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1aa969-5c90-4416-8d7d-26800ade5e70 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:02.621018 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1aa969-5c90-4416-8d7d-26800ade5e70 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:02.621294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1aa969-5c90-4416-8d7d-26800ade5e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:02.644494 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6e1aa969-5c90-4416-8d7d-26800ade5e70 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:23:02.645858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6e1aa969-5c90-4416-8d7d-26800ade5e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:02.655906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1384/2767] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:23:02.660495 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bffcb-2f46-4dc9-ba47-7ffe97a23670 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:02.672403 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bffcb-2f46-4dc9-ba47-7ffe97a23670 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:02.677898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:02.678637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:02.678922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:02.679311 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:02.679951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:23:02.680279 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:02.682185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:02.694667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1384/2768] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:23:02.699517 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1aa969-5c90-4416-8d7d-26800ade5e70 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:02.710189 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1aa969-5c90-4416-8d7d-26800ade5e70 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:02.715009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:02.716289 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:02.716627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:02.716964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:02.717621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:02.733176 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:02.982287 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1385/2769] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:23:02 2022] POST /identity/v3/projects => generated 373 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:23:02.986601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:02.996500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf16d3-0717-4cb2-bfc8-2506558a61a4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:03.052810 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:03.053413 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:03.053737 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:03.054013 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:03.054642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:03.056549 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:03.140710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1385/2770] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:23:02 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:23:03.144816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:03.155691 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038c596-e58e-4cc6-8829-736ad1a8daf0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:03.160362 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:03.160982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:03.161266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:03.161592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:03.162242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:03.163852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:03.174437 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1386/2771] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:23:03.178269 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:03.188540 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a8e0f-d66b-4fd2-b160-8dbb37bfae5b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:03.193035 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be342172-eb43-4d2d-9c43-50dae152b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:03.193647 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be342172-eb43-4d2d-9c43-50dae152b246 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:03.193930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be342172-eb43-4d2d-9c43-50dae152b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:03.194202 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be342172-eb43-4d2d-9c43-50dae152b246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:03.194829 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be342172-eb43-4d2d-9c43-50dae152b246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:03.196406 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be342172-eb43-4d2d-9c43-50dae152b246 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:03.207207 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1386/2772] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:23:03.211151 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:03.221131 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bfc48-930d-4323-a16b-2f48ca1382ec admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:03.225857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:03.226497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:03.226839 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:03.227119 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] PATH_INFO: `/v3/projects/af16e8b4a7ef4f66bfbce95b53c31a8a/users/dd025c18bec54d449e6fd89fa6376995/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:03.236285 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:03.242797 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af16e8b4a7ef4f66bfbce95b53c31a8a, user_id=dd025c18bec54d449e6fd89fa6376995, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:03.245433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:03.363843 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1387/2773] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:23:03 2022] PUT /identity/v3/projects/af16e8b4a7ef4f66bfbce95b53c31a8a/users/dd025c18bec54d449e6fd89fa6376995/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:23:03.376026 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4bf5-140d-41de-a955-dfec208294e8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:03.376310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4bf5-140d-41de-a955-dfec208294e8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:03.376585 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4bf5-140d-41de-a955-dfec208294e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:03.405225 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7a8f4bf5-140d-41de-a955-dfec208294e8 None None] MFA Rules not processed for user `dd025c18bec54d449e6fd89fa6376995`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:23:03.406452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7a8f4bf5-140d-41de-a955-dfec208294e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:03.411838 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:03.470424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1387/2774] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:23:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:23:03.481058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:03.483393 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:03.541858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3ad4a-2bc0-42a7-b91a-1ae6dfb26027 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:03.545905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a0083-ec77-48a9-9753-b0acff7e0b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:03.546522 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a0083-ec77-48a9-9753-b0acff7e0b5e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:03.546804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a0083-ec77-48a9-9753-b0acff7e0b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:03.547215 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a0083-ec77-48a9-9753-b0acff7e0b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:03.555703 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a0083-ec77-48a9-9753-b0acff7e0b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:03.557161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a0083-ec77-48a9-9753-b0acff7e0b5e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:03.566282 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1388/2775] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:23:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:23:13.511834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=24 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:13.562067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:14.837310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f4bf5-140d-41de-a955-dfec208294e8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:14.912205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f4bf5-140d-41de-a955-dfec208294e8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:14.917084 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:14.917862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:14.918135 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:14.918404 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] PATH_INFO: `/v3/users/dd025c18bec54d449e6fd89fa6376995` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:14.919757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd025c18bec54d449e6fd89fa6376995)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:14.922711 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:15.081867 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd025c18bec54d449e6fd89fa6376995'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:15.082937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd025c18bec54d449e6fd89fa6376995'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:15.104840 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd025c18bec54d449e6fd89fa6376995'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:15.159310 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd025c18bec54d449e6fd89fa6376995'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:15.159927 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd025c18bec54d449e6fd89fa6376995'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:15.172534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1388/2776] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:23:14 2022] DELETE /identity/v3/users/dd025c18bec54d449e6fd89fa6376995 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:23:15.178210 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:15.178683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:15.178783 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:15.198269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:23:15.204801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:15.258840 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1389/2777] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:23:15.268593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:15.270105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:15.327721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1aaba-d9bc-440c-a7c2-4d5a876eb4aa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:15.331470 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:15.332129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:15.332960 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:15.333236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:15.342793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:15.344229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:15.353863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1389/2778] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:23:15.793845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:15.795450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:15.808806 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5982951-0527-41dc-ba33-58dd50537f2a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:15.818027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:15.818027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:15.818027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:15.818027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] PATH_INFO: `/v3/projects/af16e8b4a7ef4f66bfbce95b53c31a8a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:15.820161 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af16e8b4a7ef4f66bfbce95b53c31a8a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:15.822669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:15.937837 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af16e8b4a7ef4f66bfbce95b53c31a8a', 'request_id': 'req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:16.005526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af16e8b4a7ef4f66bfbce95b53c31a8a', 'request_id': 'req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:16.011531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af16e8b4a7ef4f66bfbce95b53c31a8a', 'request_id': 'req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:16.022692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] The token cache is being invalidate because project af16e8b4a7ef4f66bfbce95b53c31a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:23:16.022981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:16.023615 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:23:16.025196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1390/2779] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:23:15 2022] DELETE /identity/v3/projects/af16e8b4a7ef4f66bfbce95b53c31a8a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:23:25.349642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:26.019538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:23:35.021850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:23:35.023326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:35.034993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd3b38-c48b-4276-9ec1-398e1bff38fb service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:23:35.084967 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:23:35.085582 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:23:35.085862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:23:35.086133 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:23:35.087824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:23:35.156210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:23:35.157683 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:23:35.168512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1390/2780] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:23:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:23:45.164199 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=54 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:24:08.433737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:24:08.435171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:08.492700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7ee7c-82e1-4bbd-91ab-a7af14d63b34 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:24:08.497026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:24:08.497679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:24:08.497962 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:24:08.498236 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:24:08.499943 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:08.513513 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:24:08.514949 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:24:08.525692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1391/2781] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:24:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:24:18.520830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=26 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:24:21.201374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:24:21.203074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:21.214161 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a240-774f-4fe5-9c4e-dc22e0ae0e74 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:24:21.217949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:24:21.218550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:24:21.218825 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:24:21.219096 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:24:21.220777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:21.235039 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:24:21.236472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:24:21.245256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1391/2782] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:24:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:24:31.240801 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:24:39.204613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:24:39.204613 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:39.219187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a61ab-21f4-42a6-bc3e-96147fbf1f1b service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:24:39.226165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:24:39.227611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:24:39.227611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:24:39.227611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:24:39.229331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:39.242915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:24:39.244787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:24:39.257208 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1392/2783] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:24:41.750628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:24:41.763774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f1286-92a6-4ec5-b7c9-fece944d49db service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:24:41.769283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:24:41.769911 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:24:41.770196 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:24:41.770474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:24:41.772181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:41.785097 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:24:41.786710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:24:41.798319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1392/2784] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:24:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:24:49.252414 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:24:51.793377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:24:52.843413 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:24:52.852891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8327f-ef98-49d8-a474-248e6caadaff service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:24:52.856680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:24:52.857277 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:24:52.857558 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:24:52.857866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:24:52.859568 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:24:52.927701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:24:52.929140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:24:52.939242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1393/2785] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:25:00.446632 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:25:00.449050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:00.470766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62e8e4-bde3-4b92-bbb5-51ec68e5485a service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:25:00.482767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:25:00.485537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:25:00.485537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:25:00.485537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] PATH_INFO: `/v3/users/b694f1163ef54b82b86e06bf825c57e4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:25:00.486559 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b694f1163ef54b82b86e06bf825c57e4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:25:00.488640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:25:00.689806 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b694f1163ef54b82b86e06bf825c57e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:00.690577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b694f1163ef54b82b86e06bf825c57e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:00.704937 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b694f1163ef54b82b86e06bf825c57e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:00.782526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b694f1163ef54b82b86e06bf825c57e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:00.783146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b694f1163ef54b82b86e06bf825c57e4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:00.795739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1393/2786] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:25:00 2022] DELETE /identity/v3/users/b694f1163ef54b82b86e06bf825c57e4 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:25:01.056335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:25:01.074223 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb95c1-4f89-4a1a-849f-1c123bf4eeba service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:25:01.131641 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:25:01.132290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:25:01.132574 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:25:01.132847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] PATH_INFO: `/v3/projects/56e82ba24a474557af758826743136d9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:25:01.134039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e82ba24a474557af758826743136d9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:25:01.136442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:25:01.228570 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e82ba24a474557af758826743136d9', 'request_id': 'req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.319160 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e82ba24a474557af758826743136d9', 'request_id': 'req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.328750 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e82ba24a474557af758826743136d9', 'request_id': 'req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.341080 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] The token cache is being invalidate because project 56e82ba24a474557af758826743136d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:25:01.341632 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.342585 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.344104 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1394/2787] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:25:01 2022] DELETE /identity/v3/projects/56e82ba24a474557af758826743136d9 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:25:01.350765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:25:01.352484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:01.367424 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2cce7-4910-4e72-b8de-ca6502b1eae1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:25:01.427852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:25:01.427852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:25:01.427852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:25:01.428428 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] PATH_INFO: `/v3/users/c75af86aac424ddda3c8afd3ba30b4d5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:25:01.429203 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75af86aac424ddda3c8afd3ba30b4d5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:25:01.431169 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:25:01.627733 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75af86aac424ddda3c8afd3ba30b4d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.628456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75af86aac424ddda3c8afd3ba30b4d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.641556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75af86aac424ddda3c8afd3ba30b4d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.713512 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75af86aac424ddda3c8afd3ba30b4d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.714758 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c9878bf-e909-4d5b-a02b-2681298b5b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75af86aac424ddda3c8afd3ba30b4d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:01.717205 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:25:01.727762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1394/2788] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:25:01 2022] DELETE /identity/v3/users/c75af86aac424ddda3c8afd3ba30b4d5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:25:02.075415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:25:02.076963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:02.097422 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee2dbd-114e-4a9a-a448-e7a52d9a8f8f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:25:02.125796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:25:02.164526 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:25:02.165336 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:25:02.165722 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:25:02.166095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] PATH_INFO: `/v3/projects/89ffdbbdedc24fbca2fbb10c2d2fd466` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:25:02.168040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ffdbbdedc24fbca2fbb10c2d2fd466)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:25:02.170138 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:25:02.280829 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ffdbbdedc24fbca2fbb10c2d2fd466', 'request_id': 'req-4e727392-562b-4bb5-820e-a99b17b0795e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:02.376868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ffdbbdedc24fbca2fbb10c2d2fd466', 'request_id': 'req-4e727392-562b-4bb5-820e-a99b17b0795e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:02.384518 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ffdbbdedc24fbca2fbb10c2d2fd466', 'request_id': 'req-4e727392-562b-4bb5-820e-a99b17b0795e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:02.396158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] The token cache is being invalidate because project 89ffdbbdedc24fbca2fbb10c2d2fd466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:25:02.396478 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:02.397174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:25:02.398684 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1395/2789] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:25:02 2022] DELETE /identity/v3/projects/89ffdbbdedc24fbca2fbb10c2d2fd466 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:25:04.070012 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:25:04.070528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:25:04.070528 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:25:04.090886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] MFA Rules not processed for user `b1fa7bf738264e77b868ce9d7863214e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:25:04.092105 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:04.148641 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1395/2790] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:25:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:25:06.870956 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:25:06.872395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:06.941961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e727392-562b-4bb5-820e-a99b17b0795e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:25:06.946928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:25:06.947531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:25:06.947878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:25:06.948719 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:25:06.950721 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:06.962496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:25:06.964757 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:25:06.975126 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1396/2791] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:25:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:25:14.141993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=29 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:25:16.952884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:25:16.954593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:16.970714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:25:17.046884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b21fc-7e1d-4519-9836-22236294b79c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:25:17.052390 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46fd540df66143b2b768cc8d9f4beed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:25:17.053008 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:25:17.057746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:25:17.057746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:25:17.057746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:25:17.068763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:25:17.070455 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:25:17.082153 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1396/2792] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:25:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:25:27.077216 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=34 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:31.588042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:31.598053 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df12e3-2d6f-44ec-a218-4d0397fac78e service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:31.603171 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:31.603805 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:31.604088 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:31.604358 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:31.606219 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:31.618678 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:31.620251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:31.630582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1397/2793] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:31.703643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:31.704982 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:31.772407 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881b886-88bf-41aa-9ef6-8fd97880a2a6 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:31.776959 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:31.777603 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:31.777887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:31.778164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] PATH_INFO: `/v3/users/b1fa7bf738264e77b868ce9d7863214e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:31.779309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1fa7bf738264e77b868ce9d7863214e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:31.782347 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:31.938983 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1fa7bf738264e77b868ce9d7863214e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:31.939590 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1fa7bf738264e77b868ce9d7863214e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:31.951459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1fa7bf738264e77b868ce9d7863214e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.004284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1fa7bf738264e77b868ce9d7863214e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.004900 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ad12916b-5cb5-4204-8360-ce99b46bbf5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1fa7bf738264e77b868ce9d7863214e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.017950 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1397/2794] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:26:31 2022] DELETE /identity/v3/users/b1fa7bf738264e77b868ce9d7863214e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:26:32.270141 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:32.281041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a88a91-ef12-4692-aa61-655bd4ac6f64 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:32.340890 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:32.341519 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.341830 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.342107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] PATH_INFO: `/v3/projects/6dde43a19e95472d87ea4273922156be` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.343229 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dde43a19e95472d87ea4273922156be)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:32.345594 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:32.451087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dde43a19e95472d87ea4273922156be', 'request_id': 'req-44a28ea3-fe86-4602-95c2-8b1339c59048'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.502163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dde43a19e95472d87ea4273922156be', 'request_id': 'req-44a28ea3-fe86-4602-95c2-8b1339c59048'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.508009 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dde43a19e95472d87ea4273922156be', 'request_id': 'req-44a28ea3-fe86-4602-95c2-8b1339c59048'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.527453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] The token cache is being invalidate because project 6dde43a19e95472d87ea4273922156be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:26:32.527453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.527453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-44a28ea3-fe86-4602-95c2-8b1339c59048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:32.527453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1398/2795] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:26:32 2022] DELETE /identity/v3/projects/6dde43a19e95472d87ea4273922156be => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:32.528503 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83cfa-5efe-441c-9ae2-fcba90386983 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.528794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83cfa-5efe-441c-9ae2-fcba90386983 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.529181 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83cfa-5efe-441c-9ae2-fcba90386983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.549396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-44f83cfa-5efe-441c-9ae2-fcba90386983 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:32.550675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-44f83cfa-5efe-441c-9ae2-fcba90386983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:32.613620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1398/2796] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:32.621031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a68b5-12cc-4f96-9ef4-92d08d88aee0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.621708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a68b5-12cc-4f96-9ef4-92d08d88aee0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.621708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a68b5-12cc-4f96-9ef4-92d08d88aee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.641975 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-549a68b5-12cc-4f96-9ef4-92d08d88aee0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:32.643183 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-549a68b5-12cc-4f96-9ef4-92d08d88aee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:32.651095 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1399/2797] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:32.655247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83cfa-5efe-441c-9ae2-fcba90386983 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:32.665298 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83cfa-5efe-441c-9ae2-fcba90386983 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:32.669849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:32.670437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.670714 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.670987 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.672157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:26:32.672630 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:32.674255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:32.685537 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1399/2798] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:26:32.690260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a68b5-12cc-4f96-9ef4-92d08d88aee0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:32.699389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a68b5-12cc-4f96-9ef4-92d08d88aee0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:32.703979 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:32.704573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.704855 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.705129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.705792 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:32.707680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:32.795046 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1400/2799] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:26:32 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:32.799289 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:32.809634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bf997-4f03-4f53-8fbf-026bb6004379 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:32.813472 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:32.869605 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:32.870195 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.870474 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.870741 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.871374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:32.873276 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:32.943601 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1400/2800] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:26:32 2022] POST /identity/v3/users => generated 302 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:32.947759 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:32.959122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b027052-859d-4be6-ba23-be60f514f09e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:32.965869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:32.965869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.965869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.965869 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.966304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:32.967146 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:32.977930 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1401/2801] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:32.981837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:32.991697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017591f-7918-45f5-97d6-70c48a49d31c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:32.996847 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563a299-a838-444a-9358-069bb1265d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:32.997446 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563a299-a838-444a-9358-069bb1265d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:32.997892 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563a299-a838-444a-9358-069bb1265d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:32.998172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563a299-a838-444a-9358-069bb1265d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:32.998860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563a299-a838-444a-9358-069bb1265d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.000965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563a299-a838-444a-9358-069bb1265d46 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.010692 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1401/2802] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:33.014692 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:33.025054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e09c6-1052-40d2-a019-367324fb638c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:33.029618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:33.030255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.030535 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.030809 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] PATH_INFO: `/v3/projects/f203abeb05d7447182d534d8d9eb891b/users/4c2782bbfef14869a5eddcdcf4bcd9b7/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.045466 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f203abeb05d7447182d534d8d9eb891b, user_id=4c2782bbfef14869a5eddcdcf4bcd9b7, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.048221 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.106892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1402/2803] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:33 2022] PUT /identity/v3/projects/f203abeb05d7447182d534d8d9eb891b/users/4c2782bbfef14869a5eddcdcf4bcd9b7/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:33.116969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e02db-d941-4fb4-8a01-5eaf77ab0e21 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.117327 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e02db-d941-4fb4-8a01-5eaf77ab0e21 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.117638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e02db-d941-4fb4-8a01-5eaf77ab0e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.146777 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dc2e02db-d941-4fb4-8a01-5eaf77ab0e21 None None] MFA Rules not processed for user `4c2782bbfef14869a5eddcdcf4bcd9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:33.148053 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dc2e02db-d941-4fb4-8a01-5eaf77ab0e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:33.210738 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1402/2804] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:33 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:26:33.215264 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:33.220989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:33.278235 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49100c2-8eb1-4535-97de-b13d8927aa57 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:33.282914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:33.283503 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.283782 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.284054 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.284935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.286968 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.378740 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1403/2805] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:26:33 2022] POST /identity/v3/projects => generated 405 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:33.382718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e02db-d941-4fb4-8a01-5eaf77ab0e21 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:33.394164 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e02db-d941-4fb4-8a01-5eaf77ab0e21 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:33.449527 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:33.450152 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.450429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.450699 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.451343 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.453466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.577382 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1403/2806] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:26:33 2022] POST /identity/v3/users => generated 309 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:33.581424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:33.591799 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3fa79-5568-4c83-b768-d18a8d975ce6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:33.596463 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:33.597051 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.597329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.597627 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.598380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.600006 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.611849 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1404/2807] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:26:33.616098 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:33.622574 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:33.627308 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a66e6-0a3c-4d9d-a2b0-4325c7b033b5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:33.631977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:33.632628 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.632913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.633192 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] PATH_INFO: `/v3/projects/ca998412ef734630a5dabf2b7a08b279/users/5b924c1d479e4763bd31d21b05cc9c8f/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.648077 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca998412ef734630a5dabf2b7a08b279, user_id=5b924c1d479e4763bd31d21b05cc9c8f, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.650809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.759776 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1404/2808] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:33 2022] PUT /identity/v3/projects/ca998412ef734630a5dabf2b7a08b279/users/5b924c1d479e4763bd31d21b05cc9c8f/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:33.763802 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:33.774065 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48e719-6679-4e3c-b6e1-4c4b562691fa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:33.831107 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc97fbd-a6ed-48dd-bcf5-7f5d0cebaf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:33.831743 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc97fbd-a6ed-48dd-bcf5-7f5d0cebaf3a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.832026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc97fbd-a6ed-48dd-bcf5-7f5d0cebaf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.832301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc97fbd-a6ed-48dd-bcf5-7f5d0cebaf3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.832934 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc97fbd-a6ed-48dd-bcf5-7f5d0cebaf3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.834552 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc97fbd-a6ed-48dd-bcf5-7f5d0cebaf3a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.845380 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1405/2809] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:26:33.849942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:33.860985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0f32c-4daa-4fc0-8660-f09db0cc42e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:33.865985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:33.866634 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.867009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.867287 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] PATH_INFO: `/v3/projects/ca998412ef734630a5dabf2b7a08b279/users/5b924c1d479e4763bd31d21b05cc9c8f/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.869213 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca998412ef734630a5dabf2b7a08b279, user_id=5b924c1d479e4763bd31d21b05cc9c8f, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:33.871891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:33.889274 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1405/2810] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:33 2022] PUT /identity/v3/projects/ca998412ef734630a5dabf2b7a08b279/users/5b924c1d479e4763bd31d21b05cc9c8f/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:33.905711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949b23-d8de-4b6a-8b15-42f7d8b62be2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:33.906597 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949b23-d8de-4b6a-8b15-42f7d8b62be2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:33.906873 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949b23-d8de-4b6a-8b15-42f7d8b62be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:33.936712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e0949b23-d8de-4b6a-8b15-42f7d8b62be2 None None] MFA Rules not processed for user `5b924c1d479e4763bd31d21b05cc9c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:33.938586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e0949b23-d8de-4b6a-8b15-42f7d8b62be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:34.004030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:34.007411 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1406/2811] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:26:34.019526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:34.021352 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:34.080780 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ff128-6a17-441a-be80-14e647a6dfad admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:34.084451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab343a33-bcc4-45b2-ac8f-557e6b13a052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:34.085049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab343a33-bcc4-45b2-ac8f-557e6b13a052 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:34.085328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab343a33-bcc4-45b2-ac8f-557e6b13a052 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:34.085627 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab343a33-bcc4-45b2-ac8f-557e6b13a052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:34.094421 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab343a33-bcc4-45b2-ac8f-557e6b13a052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:34.098757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab343a33-bcc4-45b2-ac8f-557e6b13a052 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:34.105313 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1406/2812] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:34.241385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0949b23-d8de-4b6a-8b15-42f7d8b62be2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:34.302683 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0949b23-d8de-4b6a-8b15-42f7d8b62be2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:34.307368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:34.307999 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:34.308275 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:34.308572 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] PATH_INFO: `/v3/users/4c2782bbfef14869a5eddcdcf4bcd9b7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:34.309737 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2782bbfef14869a5eddcdcf4bcd9b7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:34.311707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:34.498725 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2782bbfef14869a5eddcdcf4bcd9b7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:34.499345 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2782bbfef14869a5eddcdcf4bcd9b7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:34.547781 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2782bbfef14869a5eddcdcf4bcd9b7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:34.548408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2782bbfef14869a5eddcdcf4bcd9b7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:34.560892 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2782bbfef14869a5eddcdcf4bcd9b7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:34.573625 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1407/2813] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:26:34 2022] DELETE /identity/v3/users/4c2782bbfef14869a5eddcdcf4bcd9b7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:34.578185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:34.578471 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:34.578746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:34.597697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:34.598884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:34.657985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1407/2814] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:34.668301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:34.669751 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:34.727424 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57901b7-80ff-49b6-b017-2ba54965f320 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:34.731048 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:34.731644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:34.731921 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:34.732190 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:34.741650 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:34.743067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:34.752383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1408/2815] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:35.172066 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:35.173518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:35.186913 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba0fc3-99ad-45ec-a153-46cb2ddedee0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:35.191637 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:35.192256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:35.192536 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:35.192811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] PATH_INFO: `/v3/projects/f203abeb05d7447182d534d8d9eb891b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:35.193971 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f203abeb05d7447182d534d8d9eb891b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:35.195890 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:35.313614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f203abeb05d7447182d534d8d9eb891b', 'request_id': 'req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.320090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f203abeb05d7447182d534d8d9eb891b', 'request_id': 'req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.326435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f203abeb05d7447182d534d8d9eb891b', 'request_id': 'req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.328922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:35.421583 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] The token cache is being invalidate because project f203abeb05d7447182d534d8d9eb891b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:26:35.421893 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.422517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.424119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1408/2816] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:26:35 2022] DELETE /identity/v3/projects/f203abeb05d7447182d534d8d9eb891b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:35.428315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:35.438097 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c6005-ba60-4563-96d0-197ae30d061e service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:35.457134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:35.503490 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:35.504349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:35.506950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:35.506950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] PATH_INFO: `/v3/users/5b924c1d479e4763bd31d21b05cc9c8f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:35.508702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b924c1d479e4763bd31d21b05cc9c8f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:35.510301 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:35.682847 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b924c1d479e4763bd31d21b05cc9c8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.683592 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b924c1d479e4763bd31d21b05cc9c8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.753373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b924c1d479e4763bd31d21b05cc9c8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.754036 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b924c1d479e4763bd31d21b05cc9c8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.766581 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-35b87a71-c174-47df-b09e-2eb3ae2de357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b924c1d479e4763bd31d21b05cc9c8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:35.778636 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1409/2817] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:26:35 2022] DELETE /identity/v3/users/5b924c1d479e4763bd31d21b05cc9c8f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:36.143642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:36.145112 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:36.160321 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2a0fd-7dba-493b-8c35-24e1e684b9d2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:36.215804 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:36.216453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.216702 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.216975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] PATH_INFO: `/v3/projects/ca998412ef734630a5dabf2b7a08b279` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.218117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca998412ef734630a5dabf2b7a08b279)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:36.220594 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:36.307241 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca998412ef734630a5dabf2b7a08b279', 'request_id': 'req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:36.312622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca998412ef734630a5dabf2b7a08b279', 'request_id': 'req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:36.319189 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca998412ef734630a5dabf2b7a08b279', 'request_id': 'req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:36.382243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] The token cache is being invalidate because project ca998412ef734630a5dabf2b7a08b279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:26:36.382540 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:36.383172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9c12660-fb85-4ceb-b4cb-61dcc811e2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:36.384669 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1409/2818] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:26:36 2022] DELETE /identity/v3/projects/ca998412ef734630a5dabf2b7a08b279 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:36.389777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408dd18-d1bd-4099-a31c-acb1867da474 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.390007 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408dd18-d1bd-4099-a31c-acb1867da474 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.390284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408dd18-d1bd-4099-a31c-acb1867da474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.409832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5408dd18-d1bd-4099-a31c-acb1867da474 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:36.411026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5408dd18-d1bd-4099-a31c-acb1867da474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:36.471308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1410/2819] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:36.479094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c593-1248-43de-ab1b-84b269a2878f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.479383 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c593-1248-43de-ab1b-84b269a2878f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.479661 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c593-1248-43de-ab1b-84b269a2878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.500129 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8245c593-1248-43de-ab1b-84b269a2878f None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:36.501344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8245c593-1248-43de-ab1b-84b269a2878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:36.509359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1410/2820] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:26:36.513771 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408dd18-d1bd-4099-a31c-acb1867da474 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:36.524423 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408dd18-d1bd-4099-a31c-acb1867da474 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:36.529207 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:36.529906 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.530174 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.530450 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.531098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:26:36.531406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:36.533754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:36.545452 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1411/2821] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:26:36.549886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c593-1248-43de-ab1b-84b269a2878f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:36.559011 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c593-1248-43de-ab1b-84b269a2878f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:36.567866 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:36.568465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.568746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.569023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.569701 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:36.571677 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:36.652579 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1411/2822] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:26:36 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:36.656747 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:36.667622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e77e-adfe-4104-978a-ca470dfcb154 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:36.694217 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:36.725524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:36.726157 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.726435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.726705 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.727330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:36.729315 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:36.817484 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1412/2823] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:26:36 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:36.821589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:36.832405 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bd3be-be80-41a5-bf89-eaae015ff744 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:36.841319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:36.841975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.842253 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.842524 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.843162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:36.844863 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:36.855502 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1412/2824] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:36.859550 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:36.869868 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa900659-42bc-4adc-8668-1d8f6bd2eadf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:36.874507 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d96ed-f50d-4d8a-82f1-5d8a9cdef8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:36.875105 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d96ed-f50d-4d8a-82f1-5d8a9cdef8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.875385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d96ed-f50d-4d8a-82f1-5d8a9cdef8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.875665 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d96ed-f50d-4d8a-82f1-5d8a9cdef8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.876861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d96ed-f50d-4d8a-82f1-5d8a9cdef8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:36.878815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d96ed-f50d-4d8a-82f1-5d8a9cdef8b6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:36.888599 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1413/2825] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:36.893533 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:36.903367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36111006-b4ef-48c2-ac9e-ab6e3aee2224 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:36.907951 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:36.908686 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:36.908965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:36.909235 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] PATH_INFO: `/v3/projects/23501fadcf894dc68dee4584be84f012/users/c0e0aaa1ddd946948cd535adbcd49a2c/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:36.923506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] RBAC: Authorizing `identity:create_grant(project_id=23501fadcf894dc68dee4584be84f012, user_id=c0e0aaa1ddd946948cd535adbcd49a2c, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:36.926179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.010765 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1413/2826] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:36 2022] PUT /identity/v3/projects/23501fadcf894dc68dee4584be84f012/users/c0e0aaa1ddd946948cd535adbcd49a2c/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:37.028154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3343c5-6f0a-4a94-bd77-aa5ca04d947f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.028444 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3343c5-6f0a-4a94-bd77-aa5ca04d947f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.028723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3343c5-6f0a-4a94-bd77-aa5ca04d947f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.056794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1f3343c5-6f0a-4a94-bd77-aa5ca04d947f None None] MFA Rules not processed for user `c0e0aaa1ddd946948cd535adbcd49a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:37.058030 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1f3343c5-6f0a-4a94-bd77-aa5ca04d947f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:37.121978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1414/2827] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:37.127185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:37.147511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:37.187423 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4121c01-15f6-4e49-a975-a087386a765f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:37.194973 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:37.195670 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.195954 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.196230 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.196902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:37.198918 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.309236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1414/2828] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:26:37 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:37.313306 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3343c5-6f0a-4a94-bd77-aa5ca04d947f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:37.325531 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3343c5-6f0a-4a94-bd77-aa5ca04d947f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:37.383269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:37.383899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.384164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.384440 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.385086 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:37.387025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.499412 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1415/2829] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:26:37 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:37.500506 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:37.515087 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e45dc4-cdad-4ad0-8bdd-23384693dbca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:37.520965 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:37.521625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.521912 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.522187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.522861 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:37.524505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.536640 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1415/2830] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:37.540909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:37.551730 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f99f5-9400-469a-9f08-7bfb482c5064 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:37.556433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:37.557074 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.557349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.557674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] PATH_INFO: `/v3/projects/7b5489fad2cc4adeb10c89d149d5f523/users/33394b70bce2402eb26cd7ba131c29e8/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.572524 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5489fad2cc4adeb10c89d149d5f523, user_id=33394b70bce2402eb26cd7ba131c29e8, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:37.575400 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.656472 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1416/2831] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:37 2022] PUT /identity/v3/projects/7b5489fad2cc4adeb10c89d149d5f523/users/33394b70bce2402eb26cd7ba131c29e8/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:37.661223 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:37.671294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9513b4-9227-4465-bdb1-86b9ffd0ad8f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:37.726718 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b087adc-a873-489d-822a-5152499496fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:37.727315 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b087adc-a873-489d-822a-5152499496fb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.727597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b087adc-a873-489d-822a-5152499496fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.727872 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b087adc-a873-489d-822a-5152499496fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.728509 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b087adc-a873-489d-822a-5152499496fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:37.730160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b087adc-a873-489d-822a-5152499496fb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.740809 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1416/2832] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:37.744910 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:37.750335 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:37.755622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4d482-c456-44d3-a7d4-95db6d280ddc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:37.760204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:37.760848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.761125 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.761396 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] PATH_INFO: `/v3/projects/7b5489fad2cc4adeb10c89d149d5f523/users/33394b70bce2402eb26cd7ba131c29e8/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.763406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5489fad2cc4adeb10c89d149d5f523, user_id=33394b70bce2402eb26cd7ba131c29e8, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:37.766042 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.775963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1417/2833] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:37 2022] PUT /identity/v3/projects/7b5489fad2cc4adeb10c89d149d5f523/users/33394b70bce2402eb26cd7ba131c29e8/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:37.787833 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e61da-572e-4030-95a7-faf00f907904 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.788121 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e61da-572e-4030-95a7-faf00f907904 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.788398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e61da-572e-4030-95a7-faf00f907904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.816660 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-485e61da-572e-4030-95a7-faf00f907904 None None] MFA Rules not processed for user `33394b70bce2402eb26cd7ba131c29e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:37.817895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-485e61da-572e-4030-95a7-faf00f907904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:37.885784 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1417/2834] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:37.897395 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:37.899102 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:37.961770 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b67f6-3727-4a21-a94f-878044e46750 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:37.965871 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5457562-0a5e-4b12-9c68-f36e09cb6570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:37.966483 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457562-0a5e-4b12-9c68-f36e09cb6570 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:37.966765 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457562-0a5e-4b12-9c68-f36e09cb6570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:37.967039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457562-0a5e-4b12-9c68-f36e09cb6570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:37.978056 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5457562-0a5e-4b12-9c68-f36e09cb6570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:37.979642 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5457562-0a5e-4b12-9c68-f36e09cb6570 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:37.990182 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1418/2835] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:38.145932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e61da-572e-4030-95a7-faf00f907904 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:38.163846 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:38.209141 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e61da-572e-4030-95a7-faf00f907904 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:38.213682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:38.214302 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:38.214580 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:38.214850 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] PATH_INFO: `/v3/users/c0e0aaa1ddd946948cd535adbcd49a2c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:38.216056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e0aaa1ddd946948cd535adbcd49a2c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:38.218009 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:38.385249 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e0aaa1ddd946948cd535adbcd49a2c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:38.385949 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e0aaa1ddd946948cd535adbcd49a2c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:38.398165 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e0aaa1ddd946948cd535adbcd49a2c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:38.479364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e0aaa1ddd946948cd535adbcd49a2c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:38.479990 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e0aaa1ddd946948cd535adbcd49a2c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:38.492566 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1418/2836] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:26:38 2022] DELETE /identity/v3/users/c0e0aaa1ddd946948cd535adbcd49a2c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:38.497020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:38.497313 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:38.497616 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:38.517115 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:38.518954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:38.577404 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1419/2837] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:38.587834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:38.589183 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:38.645493 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795da4e-f0e4-46d6-a767-eeba3ad9b2e3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:38.649119 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:38.649748 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:38.650033 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:38.650307 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:38.659338 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:38.661056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:38.670441 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1419/2838] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:39.075024 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:39.076479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:39.089629 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7596e-572b-4a14-8475-0c9ade7a7768 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:39.094392 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:39.095031 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:39.095340 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:39.095612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] PATH_INFO: `/v3/projects/23501fadcf894dc68dee4584be84f012` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:39.097251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=23501fadcf894dc68dee4584be84f012)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:39.099188 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:39.228041 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23501fadcf894dc68dee4584be84f012', 'request_id': 'req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.301994 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23501fadcf894dc68dee4584be84f012', 'request_id': 'req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.307696 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23501fadcf894dc68dee4584be84f012', 'request_id': 'req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.319319 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] The token cache is being invalidate because project 23501fadcf894dc68dee4584be84f012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:26:39.323620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.323620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.323620 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1420/2839] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:26:39 2022] DELETE /identity/v3/projects/23501fadcf894dc68dee4584be84f012 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:39.325924 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:39.336685 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f12664-a93c-46a1-bab9-ce1c80fd790f service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:39.371117 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:39.393656 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:39.394299 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:39.394577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:39.394849 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] PATH_INFO: `/v3/users/33394b70bce2402eb26cd7ba131c29e8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:39.395962 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33394b70bce2402eb26cd7ba131c29e8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:39.397979 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:39.641988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33394b70bce2402eb26cd7ba131c29e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.642577 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33394b70bce2402eb26cd7ba131c29e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.654497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33394b70bce2402eb26cd7ba131c29e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.724316 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33394b70bce2402eb26cd7ba131c29e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.724940 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4ee5014a-2089-4cfe-9680-92119c9f6b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33394b70bce2402eb26cd7ba131c29e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:39.738437 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1420/2840] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:26:39 2022] DELETE /identity/v3/users/33394b70bce2402eb26cd7ba131c29e8 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:26:40.200866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:40.204794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:40.215802 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f96ae6-add7-45ae-b1a2-3d2e900fd6fd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:40.228168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:40.272917 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:40.273573 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.273865 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.274142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] PATH_INFO: `/v3/projects/7b5489fad2cc4adeb10c89d149d5f523` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.275273 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5489fad2cc4adeb10c89d149d5f523)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:40.277184 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:40.388982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5489fad2cc4adeb10c89d149d5f523', 'request_id': 'req-c6cadb2f-a2a3-41b7-8360-38f021da78ac'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:40.445418 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5489fad2cc4adeb10c89d149d5f523', 'request_id': 'req-c6cadb2f-a2a3-41b7-8360-38f021da78ac'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:40.451816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5489fad2cc4adeb10c89d149d5f523', 'request_id': 'req-c6cadb2f-a2a3-41b7-8360-38f021da78ac'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:40.464363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] The token cache is being invalidate because project 7b5489fad2cc4adeb10c89d149d5f523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:26:40.464815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:40.465448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c6cadb2f-a2a3-41b7-8360-38f021da78ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:26:40.466996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1421/2841] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:26:40 2022] DELETE /identity/v3/projects/7b5489fad2cc4adeb10c89d149d5f523 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:40.472541 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71380042-4fe8-4777-8737-fcb47402bb5d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.473057 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71380042-4fe8-4777-8737-fcb47402bb5d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.473160 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71380042-4fe8-4777-8737-fcb47402bb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.493657 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-71380042-4fe8-4777-8737-fcb47402bb5d None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:40.496233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-71380042-4fe8-4777-8737-fcb47402bb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:40.555520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1421/2842] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:40.563891 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf40a-b670-495c-ab79-5a72b8d37ec9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.564149 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf40a-b670-495c-ab79-5a72b8d37ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.564421 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf40a-b670-495c-ab79-5a72b8d37ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.584726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9a2cf40a-b670-495c-ab79-5a72b8d37ec9 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:40.585954 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9a2cf40a-b670-495c-ab79-5a72b8d37ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:40.594014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1422/2843] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:40.599187 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71380042-4fe8-4777-8737-fcb47402bb5d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:40.609005 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71380042-4fe8-4777-8737-fcb47402bb5d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:40.614429 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:40.615024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.615344 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.615620 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.616261 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:26:40.616562 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:40.618198 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:40.629746 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1422/2844] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:26:40.634269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cf40a-b670-495c-ab79-5a72b8d37ec9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:40.643964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cf40a-b670-495c-ab79-5a72b8d37ec9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:40.648605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:40.649198 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.649486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.649788 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.650416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:40.652564 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:40.733409 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1423/2845] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:26:40 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:40.736770 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:40.747725 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6893c5d-a8c0-418a-8348-afd5e228a7c5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:40.788415 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:40.803732 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:40.804334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.804614 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.804963 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.805698 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:40.807684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:40.907543 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1423/2846] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:26:40 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:26:40.912348 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:40.922679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a687feb-7884-4493-932f-5559ef3820fe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:40.927676 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:40.928269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.928547 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.928816 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.929447 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:40.931064 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:40.945502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1424/2847] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:40.949687 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:40.960260 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc11f4a-4f12-4d3c-b6c6-b53b807a3edb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:40.965060 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb915c-f95b-49f5-bde9-7e42aefdf455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:40.965877 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb915c-f95b-49f5-bde9-7e42aefdf455 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:40.966484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb915c-f95b-49f5-bde9-7e42aefdf455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:40.966766 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb915c-f95b-49f5-bde9-7e42aefdf455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:40.967467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb915c-f95b-49f5-bde9-7e42aefdf455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:40.969139 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb915c-f95b-49f5-bde9-7e42aefdf455 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:40.980507 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1424/2848] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:40.984677 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:40.994682 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831e81c-b8cc-40f9-8fdc-e74bc87317f6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:40.999330 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:40.999964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:41.000242 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:41.000523 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] PATH_INFO: `/v3/projects/da817fbbed5b4f5e96f2a996567f1701/users/ff624529c1e54976b4540f8ab39f3cce/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:41.014989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da817fbbed5b4f5e96f2a996567f1701, user_id=ff624529c1e54976b4540f8ab39f3cce, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:41.017674 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:41.096098 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1425/2849] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:40 2022] PUT /identity/v3/projects/da817fbbed5b4f5e96f2a996567f1701/users/ff624529c1e54976b4540f8ab39f3cce/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:41.101739 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f913f-3c0c-47e4-9030-877184fa8d75 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:41.102029 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f913f-3c0c-47e4-9030-877184fa8d75 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:41.102303 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f913f-3c0c-47e4-9030-877184fa8d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:41.120874 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9c9f913f-3c0c-47e4-9030-877184fa8d75 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:41.122071 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9c9f913f-3c0c-47e4-9030-877184fa8d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:41.179430 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1425/2850] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:41.189154 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:41.191560 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:41.236796 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:41.248029 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a29f23-dc4f-4ef4-b5b1-c762f1078f22 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:41.251950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:41.252563 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:41.252842 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:41.253130 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:41.262815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:41.264268 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:41.275040 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1426/2851] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:42.474942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e444e2-55d4-4038-b8e4-ab5bb249897c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:42.478459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e444e2-55d4-4038-b8e4-ab5bb249897c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:42.478459 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e444e2-55d4-4038-b8e4-ab5bb249897c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:42.510751 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-86e444e2-55d4-4038-b8e4-ab5bb249897c None None] MFA Rules not processed for user `ff624529c1e54976b4540f8ab39f3cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:42.511991 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-86e444e2-55d4-4038-b8e4-ab5bb249897c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:42.586695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1426/2852] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:26:42.591300 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:42.602919 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f079b-878d-4b4e-8b0f-0db7db0e0d8b service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:42.607606 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:42.608238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:42.608520 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:42.608795 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:42.609459 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:42.611408 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:42.685914 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1427/2853] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:26:42 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:26:42.690050 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e444e2-55d4-4038-b8e4-ab5bb249897c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:42.701333 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e444e2-55d4-4038-b8e4-ab5bb249897c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:42.747452 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:42.756781 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:42.757379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:42.757753 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:42.758025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:42.758645 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:42.760631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:42.838283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1427/2854] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:26:42 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:26:42.845434 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:42.850940 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fdaba-905f-468c-8469-b697c45da1d4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:42.856011 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:42.856623 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:42.856904 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:42.857175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:42.857851 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:42.859438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:42.870436 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1428/2855] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:42.874616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:42.885675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc1f72-2647-4b44-9996-b236a4a404d2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:42.891229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:42.891229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:42.891229 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:42.891556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] PATH_INFO: `/v3/projects/ce77c813b4434653b7933ed9ef4b7631/users/e18e98c024c9480d84b309992c0a44b2/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:42.907500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce77c813b4434653b7933ed9ef4b7631, user_id=e18e98c024c9480d84b309992c0a44b2, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:42.910844 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:42.999209 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1428/2856] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:42 2022] PUT /identity/v3/projects/ce77c813b4434653b7933ed9ef4b7631/users/e18e98c024c9480d84b309992c0a44b2/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:26:43.003027 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:43.013679 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f90cbe-fdde-4350-a196-e8846282d66e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:43.070322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b98ace-976b-47e6-9055-99a93dadbcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:43.070926 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b98ace-976b-47e6-9055-99a93dadbcd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:43.071209 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b98ace-976b-47e6-9055-99a93dadbcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:43.071487 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b98ace-976b-47e6-9055-99a93dadbcd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:43.072122 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b98ace-976b-47e6-9055-99a93dadbcd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:43.073784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b98ace-976b-47e6-9055-99a93dadbcd5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:43.083288 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1429/2857] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:26:43.087419 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:43.098017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a533-6dc8-4e15-a88f-c485d7353c21 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:43.102716 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:43.103368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:43.103652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:43.103931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] PATH_INFO: `/v3/projects/ce77c813b4434653b7933ed9ef4b7631/users/e18e98c024c9480d84b309992c0a44b2/roles/09a611f233ce4fff97af2ed34e916c22` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:43.105942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce77c813b4434653b7933ed9ef4b7631, user_id=e18e98c024c9480d84b309992c0a44b2, role_id=09a611f233ce4fff97af2ed34e916c22)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:43.108599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:43.119835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1429/2858] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:26:43 2022] PUT /identity/v3/projects/ce77c813b4434653b7933ed9ef4b7631/users/e18e98c024c9480d84b309992c0a44b2/roles/09a611f233ce4fff97af2ed34e916c22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:26:44.359364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:44.359938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:44.359938 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:44.388726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] MFA Rules not processed for user `e18e98c024c9480d84b309992c0a44b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:26:44.389972 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:44.460152 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1430/2859] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:26:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:26:44.471454 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:44.474367 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:44.537841 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724871c-f4dd-45d3-b867-bf42e37eb271 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:44.541599 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:44.542237 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:44.542520 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:44.542795 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:44.598442 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:44.599891 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:44.608597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1430/2860] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:46.292950 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:46.294351 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:46.306087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f533a26-5246-4930-93d4-bdb31814f2cf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:46.309546 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:46.310165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:46.310443 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:46.310713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:46.319866 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:46.321278 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:46.330720 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1431/2861] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:26:54.604675 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:56.326580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:26:57.120816 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:26:57.122281 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:57.191542 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576f29-79b3-49fb-a3b7-8fca1f33bca5 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:26:57.197094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:26:57.197734 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:26:57.198019 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:26:57.198297 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:26:57.200049 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:26:57.212227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:26:57.213898 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:26:57.223682 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1431/2862] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:26:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:01.362211 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:01.373794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319ba71-0757-4b03-aa91-92c7bfd13755 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:01.379254 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:01.379846 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:01.380129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:01.380401 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:01.382256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:01.394516 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:01.396083 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:01.406787 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1432/2863] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:02.121244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:02.184243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c34914-dd60-4a0f-9343-5a4bbc0eb29c service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:02.188834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:02.189462 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:02.189778 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:02.190055 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] PATH_INFO: `/v3/users/ff624529c1e54976b4540f8ab39f3cce` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:02.191182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff624529c1e54976b4540f8ab39f3cce)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:02.193090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:02.357360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff624529c1e54976b4540f8ab39f3cce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.358063 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff624529c1e54976b4540f8ab39f3cce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.370774 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff624529c1e54976b4540f8ab39f3cce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.427852 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff624529c1e54976b4540f8ab39f3cce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.428489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff624529c1e54976b4540f8ab39f3cce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.442255 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1432/2864] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:27:02 2022] DELETE /identity/v3/users/ff624529c1e54976b4540f8ab39f3cce => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:02.699389 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:02.710230 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d169ee-be10-4e19-83b8-016b18faa7e4 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:02.772634 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:02.773260 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:02.773539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:02.773905 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] PATH_INFO: `/v3/projects/da817fbbed5b4f5e96f2a996567f1701` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:02.775645 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da817fbbed5b4f5e96f2a996567f1701)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:02.777515 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:02.901859 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da817fbbed5b4f5e96f2a996567f1701', 'request_id': 'req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.970702 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da817fbbed5b4f5e96f2a996567f1701', 'request_id': 'req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.976763 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da817fbbed5b4f5e96f2a996567f1701', 'request_id': 'req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.989391 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] The token cache is being invalidate because project da817fbbed5b4f5e96f2a996567f1701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:27:02.989710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.990373 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:02.991964 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1433/2865] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:27:02 2022] DELETE /identity/v3/projects/da817fbbed5b4f5e96f2a996567f1701 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:02.996534 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:02.997993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:03.011078 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b374-3e38-4004-bfa5-3854b64e307b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:03.015708 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:03.074254 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:03.074902 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:03.075182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:03.075460 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] PATH_INFO: `/v3/users/e18e98c024c9480d84b309992c0a44b2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:03.076597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18e98c024c9480d84b309992c0a44b2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:03.078643 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:03.241243 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18e98c024c9480d84b309992c0a44b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.241932 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18e98c024c9480d84b309992c0a44b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.254116 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18e98c024c9480d84b309992c0a44b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.313631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18e98c024c9480d84b309992c0a44b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.314248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9473d147-cda6-4984-b119-20b75fa4a392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18e98c024c9480d84b309992c0a44b2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.327111 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1433/2866] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:27:02 2022] DELETE /identity/v3/users/e18e98c024c9480d84b309992c0a44b2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:03.610087 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:03.615435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:03.626700 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713f4c-8a6c-444e-bf25-d4b39b8e0ef7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:03.685073 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:03.685853 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:03.686134 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:03.686403 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] PATH_INFO: `/v3/projects/ce77c813b4434653b7933ed9ef4b7631` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:03.687521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce77c813b4434653b7933ed9ef4b7631)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:03.689416 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:03.710723 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:03.799724 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce77c813b4434653b7933ed9ef4b7631', 'request_id': 'req-0951c31b-05cd-4998-a8ef-8f82f90de6ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.873986 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce77c813b4434653b7933ed9ef4b7631', 'request_id': 'req-0951c31b-05cd-4998-a8ef-8f82f90de6ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.879669 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce77c813b4434653b7933ed9ef4b7631', 'request_id': 'req-0951c31b-05cd-4998-a8ef-8f82f90de6ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.891521 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] The token cache is being invalidate because project ce77c813b4434653b7933ed9ef4b7631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:27:03.891810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.892448 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0951c31b-05cd-4998-a8ef-8f82f90de6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:03.893991 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1434/2867] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:27:03 2022] DELETE /identity/v3/projects/ce77c813b4434653b7933ed9ef4b7631 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:03.898837 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988f962-b977-49e2-a694-43135a2788f1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:03.899757 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988f962-b977-49e2-a694-43135a2788f1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:03.904227 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988f962-b977-49e2-a694-43135a2788f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:03.920570 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0988f962-b977-49e2-a694-43135a2788f1 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:03.921836 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0988f962-b977-49e2-a694-43135a2788f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:03.980895 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1434/2868] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:03.988622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e720-3fb1-4078-a4a2-4326d00a7937 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:03.988911 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e720-3fb1-4078-a4a2-4326d00a7937 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:03.989187 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e720-3fb1-4078-a4a2-4326d00a7937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.010375 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ce64e720-3fb1-4078-a4a2-4326d00a7937 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:04.011580 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ce64e720-3fb1-4078-a4a2-4326d00a7937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:04.019686 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1435/2869] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:04.023848 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988f962-b977-49e2-a694-43135a2788f1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:04.033451 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988f962-b977-49e2-a694-43135a2788f1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:04.038224 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:04.038812 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.039094 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.039364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.040042 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:27:04.040371 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:04.042025 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:04.053068 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1435/2870] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:27:04.057778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64e720-3fb1-4078-a4a2-4326d00a7937 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:04.066989 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64e720-3fb1-4078-a4a2-4326d00a7937 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:04.072435 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:04.073025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.073304 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.073596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.074243 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:04.076129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:04.184774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1436/2871] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:27:04 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:27:04.188668 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:04.199616 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834e96-f49e-4cff-9cc6-c5fdcfefd5f0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:04.211107 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:04.255625 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:04.256326 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.256496 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.256771 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.257393 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:04.260082 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:04.347909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1436/2872] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:27:04 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:27:04.352003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:04.363003 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0593f0c-2b0e-46b2-860c-734629472a86 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:04.367609 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:04.368845 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.369164 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.369445 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.370111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:04.371707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:04.381893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1437/2873] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:27:04.385930 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:04.395803 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173aac1-fbf5-4781-80b5-608722a5a6ab admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:04.400433 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30613fb-35bb-4d6b-9774-2a0f402e5129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:04.406689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30613fb-35bb-4d6b-9774-2a0f402e5129 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.406689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30613fb-35bb-4d6b-9774-2a0f402e5129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.406689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30613fb-35bb-4d6b-9774-2a0f402e5129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.406689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30613fb-35bb-4d6b-9774-2a0f402e5129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:04.406689 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30613fb-35bb-4d6b-9774-2a0f402e5129 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:04.414432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1437/2874] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:27:04.418996 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:04.428924 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbac0b-ea7c-4e27-bd05-15976239ffc8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:04.434262 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:04.434957 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.435192 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.435469 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] PATH_INFO: `/v3/projects/dbb09310f7bd470e9477eccfe40b790b/users/d1df8b23b51a4bf98bc681ce9434ec95/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.449958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb09310f7bd470e9477eccfe40b790b, user_id=d1df8b23b51a4bf98bc681ce9434ec95, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:04.452605 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:04.548559 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1438/2875] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:27:04 2022] PUT /identity/v3/projects/dbb09310f7bd470e9477eccfe40b790b/users/d1df8b23b51a4bf98bc681ce9434ec95/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:04.560750 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70266c5f-43ab-45fa-b0e7-12ac2bf26049 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.561041 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70266c5f-43ab-45fa-b0e7-12ac2bf26049 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.561319 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70266c5f-43ab-45fa-b0e7-12ac2bf26049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.589233 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-70266c5f-43ab-45fa-b0e7-12ac2bf26049 None None] MFA Rules not processed for user `d1df8b23b51a4bf98bc681ce9434ec95`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:04.590483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70266c5f-43ab-45fa-b0e7-12ac2bf26049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:04.654988 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1438/2876] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:04.667181 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:04.668858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:04.747328 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d28a7-13fd-49cc-bf24-44a37b7ae900 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:04.750878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b48ba-10b0-4edc-bb42-66f93b2b1a96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95204bd338a4a99a510a67c4ab97532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:04.751473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b48ba-10b0-4edc-bb42-66f93b2b1a96 service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:04.751754 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b48ba-10b0-4edc-bb42-66f93b2b1a96 service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:04.752025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b48ba-10b0-4edc-bb42-66f93b2b1a96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:04.760212 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b48ba-10b0-4edc-bb42-66f93b2b1a96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:04.761640 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b48ba-10b0-4edc-bb42-66f93b2b1a96 service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:04.770238 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1439/2877] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:14.652174 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:14.766405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:22.108974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70266c5f-43ab-45fa-b0e7-12ac2bf26049 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:22.170551 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70266c5f-43ab-45fa-b0e7-12ac2bf26049 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:22.175090 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:22.175710 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:22.176000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:22.176273 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] PATH_INFO: `/v3/users/d1df8b23b51a4bf98bc681ce9434ec95` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:22.177387 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1df8b23b51a4bf98bc681ce9434ec95)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:22.179361 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:22.366922 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1df8b23b51a4bf98bc681ce9434ec95'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:22.367500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1df8b23b51a4bf98bc681ce9434ec95'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:22.379244 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1df8b23b51a4bf98bc681ce9434ec95'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:22.439206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1df8b23b51a4bf98bc681ce9434ec95'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:22.439824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1df8b23b51a4bf98bc681ce9434ec95'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:22.452487 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1439/2878] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:27:22 2022] DELETE /identity/v3/users/d1df8b23b51a4bf98bc681ce9434ec95 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:22.456932 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:22.457427 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:22.457549 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:22.476536 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:22.480511 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:22.535695 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1440/2879] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:22.545114 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:22.549377 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:22.606456 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9fa1c-d151-423f-8ea0-1266fca20098 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:22.610384 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31de755-bde6-44b4-9d8a-2159c7b31d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:22.611170 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31de755-bde6-44b4-9d8a-2159c7b31d70 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:22.611681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31de755-bde6-44b4-9d8a-2159c7b31d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:22.612182 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31de755-bde6-44b4-9d8a-2159c7b31d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:22.621953 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31de755-bde6-44b4-9d8a-2159c7b31d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:22.624292 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31de755-bde6-44b4-9d8a-2159c7b31d70 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:22.634126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1440/2880] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:23.011039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.013433 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:23.024453 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aa07e-f6d4-4e3a-8f0d-704ba6a04520 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:23.030039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:23.030039 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.030329 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.030561 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] PATH_INFO: `/v3/projects/dbb09310f7bd470e9477eccfe40b790b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.031804 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb09310f7bd470e9477eccfe40b790b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:23.033794 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:23.136978 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb09310f7bd470e9477eccfe40b790b', 'request_id': 'req-40b1eaad-692f-4026-ab29-28ec9df486f3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:23.205140 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb09310f7bd470e9477eccfe40b790b', 'request_id': 'req-40b1eaad-692f-4026-ab29-28ec9df486f3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:23.211111 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb09310f7bd470e9477eccfe40b790b', 'request_id': 'req-40b1eaad-692f-4026-ab29-28ec9df486f3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:23.222774 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] The token cache is being invalidate because project dbb09310f7bd470e9477eccfe40b790b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:27:23.223059 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:23.223680 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-40b1eaad-692f-4026-ab29-28ec9df486f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:23.225160 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1441/2881] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:27:23 2022] DELETE /identity/v3/projects/dbb09310f7bd470e9477eccfe40b790b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:23.230172 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cede786-f520-43c4-8899-206f768d3281 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.230664 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cede786-f520-43c4-8899-206f768d3281 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.230792 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cede786-f520-43c4-8899-206f768d3281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.249695 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1cede786-f520-43c4-8899-206f768d3281 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:23.250907 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1cede786-f520-43c4-8899-206f768d3281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:23.268185 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:23.311526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1441/2882] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:27:23.319600 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b933c32-cd76-478e-a686-e2df6cbcdbc0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.319888 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b933c32-cd76-478e-a686-e2df6cbcdbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.320170 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b933c32-cd76-478e-a686-e2df6cbcdbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.341014 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5b933c32-cd76-478e-a686-e2df6cbcdbc0 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:23.342256 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5b933c32-cd76-478e-a686-e2df6cbcdbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:23.350142 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1442/2883] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:23.354576 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cede786-f520-43c4-8899-206f768d3281 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.364597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cede786-f520-43c4-8899-206f768d3281 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:23.369085 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:23.369713 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.370020 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.370294 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.370931 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:27:23.371236 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:23.374007 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:23.385914 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1442/2884] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:27:23.390848 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b933c32-cd76-478e-a686-e2df6cbcdbc0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.400165 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b933c32-cd76-478e-a686-e2df6cbcdbc0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:23.405213 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:23.405841 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.406124 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.406397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.407047 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:23.408923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:23.492227 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1443/2885] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:27:23 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:27:23.496295 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.506805 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce882-122e-41d8-8f82-5d7424459d50 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:23.562402 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:23.563003 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.563282 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.563556 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.564180 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:23.566140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:23.673526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1443/2886] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:27:23 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:27:23.677712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.690331 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fd404-3b9a-45f9-bc9a-9e017f42e86b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:23.696133 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:23.696738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.697020 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.697295 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.697980 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:23.699653 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:23.709838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1444/2887] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:27:23.713934 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.725703 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de498-d8a1-4a3f-be8c-fe8a1ff7aa8d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:23.730360 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939e28f-5c34-42e1-ac22-91627e684009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:23.730957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e28f-5c34-42e1-ac22-91627e684009 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.731238 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e28f-5c34-42e1-ac22-91627e684009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.731511 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e28f-5c34-42e1-ac22-91627e684009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.732631 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939e28f-5c34-42e1-ac22-91627e684009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:23.734398 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939e28f-5c34-42e1-ac22-91627e684009 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:23.744876 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1444/2888] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:27:23.749470 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.759586 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f06ef-bb80-4dd4-bfd5-871323655cf4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:23.765025 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:23.765726 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.766707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.766707 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] PATH_INFO: `/v3/projects/e069391e64764d89b3a55e96776d3319/users/fc3a0cc98af1430c99bf447d58f3509e/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.781548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e069391e64764d89b3a55e96776d3319, user_id=fc3a0cc98af1430c99bf447d58f3509e, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:23.784196 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:23.866815 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1445/2889] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:27:23 2022] PUT /identity/v3/projects/e069391e64764d89b3a55e96776d3319/users/fc3a0cc98af1430c99bf447d58f3509e/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 16:27:23.877197 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19992e-c97f-4b42-a1ab-3458f8ed3d3e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:23.878860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19992e-c97f-4b42-a1ab-3458f8ed3d3e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:23.878860 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19992e-c97f-4b42-a1ab-3458f8ed3d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:23.906158 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ea19992e-c97f-4b42-a1ab-3458f8ed3d3e None None] MFA Rules not processed for user `fc3a0cc98af1430c99bf447d58f3509e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:23.907409 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ea19992e-c97f-4b42-a1ab-3458f8ed3d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:23.971162 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1445/2890] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:23.981858 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:23.983713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:24.005093 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:24.039793 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ff161-b6e3-4c96-991d-a4c3608090f2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:24.043887 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9c3fe-7b1e-44f3-8c18-bb47e29f548d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:24.044486 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9c3fe-7b1e-44f3-8c18-bb47e29f548d service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:24.045385 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9c3fe-7b1e-44f3-8c18-bb47e29f548d service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:24.045708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9c3fe-7b1e-44f3-8c18-bb47e29f548d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:24.053981 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9c3fe-7b1e-44f3-8c18-bb47e29f548d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:24.055405 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9c3fe-7b1e-44f3-8c18-bb47e29f548d service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:24.064537 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1446/2891] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:33.707247 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19992e-c97f-4b42-a1ab-3458f8ed3d3e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:33.728843 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:33.771245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19992e-c97f-4b42-a1ab-3458f8ed3d3e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:33.776045 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:33.776667 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:33.776942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:33.777210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] PATH_INFO: `/v3/users/fc3a0cc98af1430c99bf447d58f3509e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:33.778475 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3a0cc98af1430c99bf447d58f3509e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:33.784728 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:33.966985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3a0cc98af1430c99bf447d58f3509e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:33.967655 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3a0cc98af1430c99bf447d58f3509e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:33.979466 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3a0cc98af1430c99bf447d58f3509e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.033679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3a0cc98af1430c99bf447d58f3509e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.034290 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3a0cc98af1430c99bf447d58f3509e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.046644 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1446/2892] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:27:33 2022] DELETE /identity/v3/users/fc3a0cc98af1430c99bf447d58f3509e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:34.051322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:34.051810 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:34.051925 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:34.071168 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:34.072367 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:34.132117 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1447/2893] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:27:34.142518 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:34.143978 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:34.203058 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66399e-bc1a-4523-955e-665c406254dd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:34.206793 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf13c4a-cc9d-4567-adc9-c5a02d015622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:34.207397 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13c4a-cc9d-4567-adc9-c5a02d015622 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:34.207679 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13c4a-cc9d-4567-adc9-c5a02d015622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:34.207957 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13c4a-cc9d-4567-adc9-c5a02d015622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:34.216946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf13c4a-cc9d-4567-adc9-c5a02d015622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:34.218622 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf13c4a-cc9d-4567-adc9-c5a02d015622 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:34.227926 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1447/2894] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:34.606893 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:34.609179 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:34.621657 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630287b8-287f-4897-957d-5e75b7c56f03 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:34.626687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:34.627309 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:34.627588 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:34.627857 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] PATH_INFO: `/v3/projects/e069391e64764d89b3a55e96776d3319` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:34.628988 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e069391e64764d89b3a55e96776d3319)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:34.630915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:34.721631 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e069391e64764d89b3a55e96776d3319', 'request_id': 'req-999459df-fd8a-497c-8a93-a6fbff147ed1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.805647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e069391e64764d89b3a55e96776d3319', 'request_id': 'req-999459df-fd8a-497c-8a93-a6fbff147ed1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.811533 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e069391e64764d89b3a55e96776d3319', 'request_id': 'req-999459df-fd8a-497c-8a93-a6fbff147ed1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.822618 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] The token cache is being invalidate because project e069391e64764d89b3a55e96776d3319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:27:34.822909 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.823542 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-999459df-fd8a-497c-8a93-a6fbff147ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:27:34.825026 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1448/2895] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:27:34 2022] DELETE /identity/v3/projects/e069391e64764d89b3a55e96776d3319 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:34.830996 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5be0a-0495-45ed-83e1-e736e900092c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:34.831491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5be0a-0495-45ed-83e1-e736e900092c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:34.831612 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5be0a-0495-45ed-83e1-e736e900092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:34.851969 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b2d5be0a-0495-45ed-83e1-e736e900092c None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:34.853148 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b2d5be0a-0495-45ed-83e1-e736e900092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:34.885500 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:34.911697 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1448/2896] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:34.919353 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2190fd-f57b-4568-8ef0-a4ca6a131458 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:34.919639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2190fd-f57b-4568-8ef0-a4ca6a131458 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:34.919915 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2190fd-f57b-4568-8ef0-a4ca6a131458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:34.940923 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7d2190fd-f57b-4568-8ef0-a4ca6a131458 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:34.942206 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7d2190fd-f57b-4568-8ef0-a4ca6a131458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:34.950228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1449/2897] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:34.955799 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5be0a-0495-45ed-83e1-e736e900092c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:34.965256 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5be0a-0495-45ed-83e1-e736e900092c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:34.971759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:34.972359 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:34.972638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:34.972909 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:34.973552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:27:34.973886 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:34.975465 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:34.987220 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1449/2898] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:27:34.991473 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2190fd-f57b-4568-8ef0-a4ca6a131458 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:35.001269 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2190fd-f57b-4568-8ef0-a4ca6a131458 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:35.005958 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:35.006554 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:35.006838 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:35.007114 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:35.007738 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:35.009647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:35.087442 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1450/2899] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:27:34 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:27:35.091759 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:35.102434 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b898f2b-4de7-4e77-879b-c51a2aa4a438 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:35.162140 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:35.162773 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:35.163061 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:35.163340 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:35.163974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:35.165921 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:35.255854 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1450/2900] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:27:35 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 16:27:35.260037 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:35.271461 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c228e-7192-4f38-875f-53b312d461e5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:35.276194 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:35.276867 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:35.277144 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:35.277415 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:35.278077 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:35.279649 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:35.291069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1451/2901] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:27:35.295024 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:35.305248 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15184ffd-c53a-4d0b-8e86-cc667ddafe3c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:35.309888 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59862a5-46e6-4c6a-aee5-a82105d8f18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:35.310481 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59862a5-46e6-4c6a-aee5-a82105d8f18d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:35.310767 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59862a5-46e6-4c6a-aee5-a82105d8f18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:35.311589 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59862a5-46e6-4c6a-aee5-a82105d8f18d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:35.312309 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59862a5-46e6-4c6a-aee5-a82105d8f18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:35.313942 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59862a5-46e6-4c6a-aee5-a82105d8f18d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:35.324443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1451/2902] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:27:35.329255 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:35.339294 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90635f2-896d-4ab6-8f4f-300d79f7a262 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:35.344711 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:35.345356 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:35.345670 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:35.346175 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] PATH_INFO: `/v3/projects/68aaa192d3e244f6984f14a8eb59f050/users/bb853b035f424dea91cc745a6322b377/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:35.348464 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:35.362596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68aaa192d3e244f6984f14a8eb59f050, user_id=bb853b035f424dea91cc745a6322b377, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:35.365386 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:35.428162 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1452/2903] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:27:35 2022] PUT /identity/v3/projects/68aaa192d3e244f6984f14a8eb59f050/users/bb853b035f424dea91cc745a6322b377/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:27:35.433435 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656e3a-df4c-4cff-87b3-fb0db84fe341 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:35.433763 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656e3a-df4c-4cff-87b3-fb0db84fe341 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:35.434023 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656e3a-df4c-4cff-87b3-fb0db84fe341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:35.454392 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bd656e3a-df4c-4cff-87b3-fb0db84fe341 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:35.455798 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bd656e3a-df4c-4cff-87b3-fb0db84fe341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:35.514835 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1452/2904] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:35.524379 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:35.526158 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:35.583548 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d035247-68dc-4d11-ae8f-c190d4268a6b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:35.587234 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:35.587832 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:35.588110 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:35.588383 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:35.597688 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:35.599363 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:35.609322 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1453/2905] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:36.698126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b26c-359c-4fec-8db0-5d79f6b2472b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:36.698690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b26c-359c-4fec-8db0-5d79f6b2472b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:36.698690 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b26c-359c-4fec-8db0-5d79f6b2472b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:36.728684 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9216b26c-359c-4fec-8db0-5d79f6b2472b None None] MFA Rules not processed for user `bb853b035f424dea91cc745a6322b377`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:36.729929 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9216b26c-359c-4fec-8db0-5d79f6b2472b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:36.763074 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:36.799004 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1453/2906] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:36.806477 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:36.808129 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:36.866136 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ae0cd-7813-42e7-8f70-c9195d607cb5 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:36.869825 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:36.870429 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:36.870712 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:36.870985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:36.879935 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:36.881346 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:36.890596 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1454/2907] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:38.563017 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263e07e-aa33-4aed-88ea-3137614d595a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:38.563517 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263e07e-aa33-4aed-88ea-3137614d595a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:38.563636 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263e07e-aa33-4aed-88ea-3137614d595a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:38.582374 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a263e07e-aa33-4aed-88ea-3137614d595a None None] MFA Rules not processed for user `bb853b035f424dea91cc745a6322b377`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:38.583573 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a263e07e-aa33-4aed-88ea-3137614d595a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:38.590497 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1454/2908] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:38.830052 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:38.839458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d187c9b-f224-43a4-82e3-fb4184f54626 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:38.843874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:38.844406 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:38.844687 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:38.844961 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:38.846714 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:38.856186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:38.857611 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:38.865539 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1455/2909] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:27:48.577389 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=14 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:48.862067 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:27:56.801887 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871384b4-90b8-4b3d-b86e-6f84494c041f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:56.802395 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871384b4-90b8-4b3d-b86e-6f84494c041f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:56.802508 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871384b4-90b8-4b3d-b86e-6f84494c041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:56.822138 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-871384b4-90b8-4b3d-b86e-6f84494c041f None None] MFA Rules not processed for user `bb853b035f424dea91cc745a6322b377`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:27:56.823350 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-871384b4-90b8-4b3d-b86e-6f84494c041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:56.830245 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1455/2910] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:27:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:27:57.063731 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:27:57.072982 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8965c-36bf-4a14-959d-2b8887a20209 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:27:57.076639 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:27:57.077320 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:27:57.077621 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:27:57.077897 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:27:57.079882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:27:57.089741 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:27:57.091153 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:27:57.100017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1456/2911] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:27:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:28:06.816910 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:28:07.096163 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:28:19.845834 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba207-d045-4968-a7c3-eac91535638f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:19.846368 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba207-d045-4968-a7c3-eac91535638f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:19.846483 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba207-d045-4968-a7c3-eac91535638f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:19.912067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-405ba207-d045-4968-a7c3-eac91535638f None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:28:19.913591 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-405ba207-d045-4968-a7c3-eac91535638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:28:19.976862 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1456/2912] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Sep 4 16:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:28:20.123582 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:20.125001 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:28:20.215864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb83e7f-9847-4ed6-81b6-dc2a52669aa7 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:20.221708 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:20.222308 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:20.222591 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:20.222862 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:20.230706 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:20.232222 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:20.240652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1457/2913] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:28:29.974000 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=44 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:28:30.237458 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=41 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:28:37.817794 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77729f23-5aa6-4ef5-ae3e-78d0f964698c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:37.818283 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77729f23-5aa6-4ef5-ae3e-78d0f964698c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:37.818491 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77729f23-5aa6-4ef5-ae3e-78d0f964698c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:37.837945 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-77729f23-5aa6-4ef5-ae3e-78d0f964698c None None] MFA Rules not processed for user `4fb881e220f041f1a7c99960be9aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:28:37.839126 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-77729f23-5aa6-4ef5-ae3e-78d0f964698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:28:37.846820 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1457/2914] 10.222.0.39 () {58 vars in 967 bytes} [Sun Sep 4 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:28:37.859778 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:37.870701 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa551d3b-bbc8-43ff-b681-79962ff25c1d service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:37.875652 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91bc1f3a0e9046bab7aa2c799ad46c3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:37.876251 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:37.876693 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:37.877204 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:37.885497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:37.887266 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:37.895777 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1458/2915] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Sep 4 16:28:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 16:28:40.673769 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77729f23-5aa6-4ef5-ae3e-78d0f964698c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:40.687652 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77729f23-5aa6-4ef5-ae3e-78d0f964698c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:40.692552 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:40.693206 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:40.693516 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:40.693839 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] PATH_INFO: `/v3/users/bb853b035f424dea91cc745a6322b377` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:40.695325 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb853b035f424dea91cc745a6322b377)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:40.696975 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:41.029432 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb853b035f424dea91cc745a6322b377'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.030137 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb853b035f424dea91cc745a6322b377'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.043550 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb853b035f424dea91cc745a6322b377'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.106136 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb853b035f424dea91cc745a6322b377'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.106762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-607012d7-2ee8-49b7-821c-4b4238eaca43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb853b035f424dea91cc745a6322b377'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.119318 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1458/2916] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:28:40 2022] DELETE /identity/v3/users/bb853b035f424dea91cc745a6322b377 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:28:41.365079 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:41.375878 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f462a-1fa5-4639-9ca6-872f0c905671 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:41.434882 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:41.435432 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:41.435713 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:41.435990 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] PATH_INFO: `/v3/projects/68aaa192d3e244f6984f14a8eb59f050` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:41.437101 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68aaa192d3e244f6984f14a8eb59f050)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:41.439058 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:41.577861 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68aaa192d3e244f6984f14a8eb59f050', 'request_id': 'req-4fc88887-dad6-477c-8df2-e1f469eef125'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.656745 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68aaa192d3e244f6984f14a8eb59f050', 'request_id': 'req-4fc88887-dad6-477c-8df2-e1f469eef125'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.663545 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68aaa192d3e244f6984f14a8eb59f050', 'request_id': 'req-4fc88887-dad6-477c-8df2-e1f469eef125'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.674622 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] The token cache is being invalidate because project 68aaa192d3e244f6984f14a8eb59f050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:28:41.678567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.678567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4fc88887-dad6-477c-8df2-e1f469eef125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:28:41.678567 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1459/2917] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:28:41 2022] DELETE /identity/v3/projects/68aaa192d3e244f6984f14a8eb59f050 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:28:41.682845 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72760dbe-0e9f-4ffb-83c8-52e9d933d0fe None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:41.683328 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72760dbe-0e9f-4ffb-83c8-52e9d933d0fe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:41.683445 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72760dbe-0e9f-4ffb-83c8-52e9d933d0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:41.704619 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-72760dbe-0e9f-4ffb-83c8-52e9d933d0fe None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:28:41.705905 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-72760dbe-0e9f-4ffb-83c8-52e9d933d0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:28:41.764977 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1459/2918] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:28:41.773425 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce639843-fe61-494c-9ef6-880a08f9d2f4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:41.773755 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce639843-fe61-494c-9ef6-880a08f9d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:41.774068 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce639843-fe61-494c-9ef6-880a08f9d2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:41.794063 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ce639843-fe61-494c-9ef6-880a08f9d2f4 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:28:41.795274 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ce639843-fe61-494c-9ef6-880a08f9d2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:28:41.803844 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1460/2919] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:28:41.807894 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72760dbe-0e9f-4ffb-83c8-52e9d933d0fe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:41.818067 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72760dbe-0e9f-4ffb-83c8-52e9d933d0fe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:41.822526 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:41.823113 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:41.823396 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:41.823659 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:41.824293 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 16:28:41.824593 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:41.826219 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:41.837532 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1460/2920] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Sep 4 16:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 16:28:41.841538 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce639843-fe61-494c-9ef6-880a08f9d2f4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:41.850874 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce639843-fe61-494c-9ef6-880a08f9d2f4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:41.855502 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:41.856096 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:41.856378 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:41.856651 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:41.857284 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:41.859195 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:41.973710 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1461/2921] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Sep 4 16:28:41 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:28:41.977493 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:41.987453 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb20bb-9e83-48e2-9887-a97b16f6ea37 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:42.034484 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:28:42.044146 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:42.045059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:42.045059 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:42.045305 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:42.045974 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:42.047884 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:42.160284 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1461/2922] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Sep 4 16:28:41 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 16:28:42.165106 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:42.175532 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c0448-f128-455c-be3a-513fcc016a67 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:42.180852 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:42.181438 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:42.181746 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:42.182016 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:42.182647 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:42.184228 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:42.194094 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1462/2923] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 16:28:42.198056 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:42.208952 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bae62-9bba-43d1-aee3-799bed8d991c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:42.213811 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6c1ab-9963-45fb-9c4a-32ecd5388030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:42.214410 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c1ab-9963-45fb-9c4a-32ecd5388030 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:42.214694 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c1ab-9963-45fb-9c4a-32ecd5388030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:42.214997 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c1ab-9963-45fb-9c4a-32ecd5388030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:42.215721 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c1ab-9963-45fb-9c4a-32ecd5388030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:42.217821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c1ab-9963-45fb-9c4a-32ecd5388030 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:42.229379 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1462/2924] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Sep 4 16:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 16:28:42.233364 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:42.244137 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285862e5-8017-4240-aab2-0d31ab9f1bec admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:42.248668 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:42.249305 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:42.249612 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:42.249885 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] PATH_INFO: `/v3/projects/f77bfff1b7324a0a8b961467fcf9dadb/users/e97fd8205a2b40c7bcb1c0ec3efdc16b/roles/2d1fb653600f44dda837472d2b0cfc75` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:42.264231 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77bfff1b7324a0a8b961467fcf9dadb, user_id=e97fd8205a2b40c7bcb1c0ec3efdc16b, role_id=2d1fb653600f44dda837472d2b0cfc75)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:42.266894 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:42.360496 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1463/2925] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Sep 4 16:28:42 2022] PUT /identity/v3/projects/f77bfff1b7324a0a8b961467fcf9dadb/users/e97fd8205a2b40c7bcb1c0ec3efdc16b/roles/2d1fb653600f44dda837472d2b0cfc75 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:28:42.371179 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14718a2-1f12-4116-82b8-5993f632756a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:42.371467 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14718a2-1f12-4116-82b8-5993f632756a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:42.371797 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14718a2-1f12-4116-82b8-5993f632756a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:42.400264 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b14718a2-1f12-4116-82b8-5993f632756a None None] MFA Rules not processed for user `e97fd8205a2b40c7bcb1c0ec3efdc16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:28:42.401505 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b14718a2-1f12-4116-82b8-5993f632756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:28:42.465443 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1463/2926] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 16:28:42.476630 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:28:42.478497 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:28:42.493479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:28:42.537180 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7bd12-a143-475a-bf67-1a887a15dfa8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:28:42.541109 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05692-7795-4a5a-995b-b9d916d7e990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c029d59023b04bf8b8bf9d4d5eee7390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:28:42.541734 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05692-7795-4a5a-995b-b9d916d7e990 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:28:42.542017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05692-7795-4a5a-995b-b9d916d7e990 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:28:42.542290 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05692-7795-4a5a-995b-b9d916d7e990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:28:42.550776 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05692-7795-4a5a-995b-b9d916d7e990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:28:42.552732 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05692-7795-4a5a-995b-b9d916d7e990 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:28:42.561349 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1464/2927] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:28:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:28:52.462633 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:28:52.583071 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=20 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:29:20.788674 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14718a2-1f12-4116-82b8-5993f632756a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:29:20.849638 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14718a2-1f12-4116-82b8-5993f632756a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:29:20.854597 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:29:20.855210 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:29:20.855489 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:29:20.855791 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] PATH_INFO: `/v3/users/e97fd8205a2b40c7bcb1c0ec3efdc16b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:29:20.856896 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97fd8205a2b40c7bcb1c0ec3efdc16b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:29:20.858831 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:29:21.002789 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97fd8205a2b40c7bcb1c0ec3efdc16b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.003447 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97fd8205a2b40c7bcb1c0ec3efdc16b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.015401 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97fd8205a2b40c7bcb1c0ec3efdc16b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.103681 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97fd8205a2b40c7bcb1c0ec3efdc16b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.104334 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97fd8205a2b40c7bcb1c0ec3efdc16b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.116842 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1464/2928] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Sep 4 16:29:20 2022] DELETE /identity/v3/users/e97fd8205a2b40c7bcb1c0ec3efdc16b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:29:21.121864 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:29:21.122368 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:29:21.122495 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:29:21.141069 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] MFA Rules not processed for user `f100ee5a590a4764a1d39fd503354ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 16:29:21.142298 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:29:21.201553 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1465/2929] 10.222.0.39 () {58 vars in 958 bytes} [Sun Sep 4 16:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 16:29:21.211277 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:29:21.213322 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:29:21.270186 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf622d0-bfd1-4dfd-8bf0-64c561456948 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:29:21.273824 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ef027007184b8180325d6fb26324a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:29:21.274425 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:29:21.275278 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:29:21.275558 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:29:21.284353 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:29:21.285821 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:29:21.296477 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1465/2930] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:29:21.690479 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:29:21.691985 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:29:21.703884 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdf7a4-1e7e-4d8b-bc02-871a75e8a7c5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:29:21.708963 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f100ee5a590a4764a1d39fd503354ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677669b803a2416fad6430932644a341', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:29:21.709644 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:29:21.709928 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:29:21.710200 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] PATH_INFO: `/v3/projects/f77bfff1b7324a0a8b961467fcf9dadb` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:29:21.711365 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77bfff1b7324a0a8b961467fcf9dadb)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:29:21.713784 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:29:21.831801 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77bfff1b7324a0a8b961467fcf9dadb', 'request_id': 'req-c0cbacda-563d-4cb6-aeb9-4a06345e508f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.901017 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77bfff1b7324a0a8b961467fcf9dadb', 'request_id': 'req-c0cbacda-563d-4cb6-aeb9-4a06345e508f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.906856 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77bfff1b7324a0a8b961467fcf9dadb', 'request_id': 'req-c0cbacda-563d-4cb6-aeb9-4a06345e508f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.918899 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] The token cache is being invalidate because project f77bfff1b7324a0a8b961467fcf9dadb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 16:29:21.919186 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.919813 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c0cbacda-563d-4cb6-aeb9-4a06345e508f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 16:29:21.921397 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1466/2931] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Sep 4 16:29:21 2022] DELETE /identity/v3/projects/f77bfff1b7324a0a8b961467fcf9dadb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 16:29:31.292341 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:29:31.915120 dv-cinder-852364-6-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 16:32:03.103762 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 16:32:03.105157 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:32:03.241993 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1217ce-44a2-4b36-a817-ddc07e3f5d7b service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 16:32:03.301768 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5850cc6-da48-4f7f-bdbc-d44a909ec330 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07e04ce012c849baa403dda571f7454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9b51abd49f54c64af586d43190b826e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 16:32:03.302378 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5850cc6-da48-4f7f-bdbc-d44a909ec330 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 16:32:03.302671 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5850cc6-da48-4f7f-bdbc-d44a909ec330 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 16:32:03.302946 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5850cc6-da48-4f7f-bdbc-d44a909ec330 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 16:32:03.304754 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5850cc6-da48-4f7f-bdbc-d44a909ec330 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 16:32:03.316364 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5850cc6-da48-4f7f-bdbc-d44a909ec330 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 16:32:03.317985 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5850cc6-da48-4f7f-bdbc-d44a909ec330 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 16:32:03.329047 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1466/2932] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Sep 4 16:32:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 16:32:13.324642 dv-cinder-852364-6-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}